tag:blogger.com,1999:blog-18477177849077300542024-03-13T11:51:39.716+00:00Andrew ThompsonCTO at Orbital Witness (AI Agent for real estate legal). Built Appear Here (PropTech) and the Caffè Nero app (Fintech), advises startups and VCs, founded an AI company. Ships early and often. 🚲☕️🔧⏱🚀Andrewhttp://www.blogger.com/profile/03252749023463514361noreply@blogger.comBlogger72125tag:blogger.com,1999:blog-1847717784907730054.post-79972856156170454422024-01-31T20:01:00.000+00:002024-01-31T20:01:44.900+00:00We built an AI Assistant, Orbital Copilot, that thinks like a real estate lawyer!<p> It uses the latest agentic GenAI concepts and is accelerating lawyers' work by 70%</p><p></p><div class="separator" style="clear: both; text-align: left;"><a href="https://blogger.googleusercontent.com/img/a/AVvXsEjie-ESGyjyHhfXTc0vbJmhaCTahTQknUnRRNnMK8OnGwumyNtd2a1XQ4hHw3q-dukV4FcldQOSY8dHe3BDbv3VIXO5_jUSCfBU1j1l243L-IWAXQuT57hWcxmGaG9995Xzj0ADSf3HYpQF0ArMSaqx6R3geXmripj0tzomMXc44z0qOwwK8JU-liLgWnDY" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img alt="" data-original-height="1024" data-original-width="1792" height="366" src="https://blogger.googleusercontent.com/img/a/AVvXsEjie-ESGyjyHhfXTc0vbJmhaCTahTQknUnRRNnMK8OnGwumyNtd2a1XQ4hHw3q-dukV4FcldQOSY8dHe3BDbv3VIXO5_jUSCfBU1j1l243L-IWAXQuT57hWcxmGaG9995Xzj0ADSf3HYpQF0ArMSaqx6R3geXmripj0tzomMXc44z0qOwwK8JU-liLgWnDY=w640-h366" width="640" /></a></div><br /><br /><p></p><p><br /></p><p><br /></p><p><br /></p><p><br /></p><p><br /></p><p><br /></p><p><br /></p><p><br /></p><p><br /></p><h2 style="text-align: left;"><div style="font-size: medium; font-weight: 400; text-align: left;"><br /></div><p style="font-size: medium; font-weight: 400;"></p><div style="text-align: left;"><span style="font-size: small; font-weight: normal;">(This was re-posted from a blog post I recently wrote on <a href="https://tech.orbitalwitness.com/posts/2024-01-10-we-built-an-ai-agent-that-thinks-like-a-real-estate-lawyer/">Orbital Witness's Tech Blog</a> just after we launched Orbital Copilot)</span></div><hr /></h2><h2 style="text-align: left;">Executive Summary</h2><p></p><ul style="text-align: left;"><li><b>Innovative Partnership</b>: Orbital Witness and Bryan Cave Leighton Paisner (BCLP) collaborate to bring <a href="https://www.orbitalwitness.com/copilot">Orbital Copilot</a> to the forefront of real estate legal practice</li></ul><ul style="text-align: left;"><li><b>Evolution of AI in LegalTech</b>: The transition from classical machine learning (ML) to Large Language Models (LLMs) and now the rise of AI Agents transforms what is possible with Generative AI</li></ul><ul style="text-align: left;"><li><b>Orbital Copilot’s Capabilities</b>: Offers unparalleled analysis, review, and reporting for real estate legal documents, giving lawyers up to 70% in time savings for lease reporting and other tasks</li></ul><ul style="text-align: left;"><li><b>Global Expansion</b>: Following the UK success, Orbital Copilot will expand to the U.S. and other sophisticated real estate markets</li></ul><ul style="text-align: left;"><li><b>Customer Cohort</b>: Prestigious law firms are among the first adopters, demonstrating trust and confidence in Orbital Copilot</li></ul><ul style="text-align: left;"><li><b>Let’s build the future together</b>: Limited spots on our <a href="https://www.orbitalwitness.com/copilot">early adopters waitlist</a> are available</li></ul><div><br /></div><div><h2 style="text-align: left;">Introduction</h2><div><br /></div><div>Two months ago we announced our <a href="https://www.bclplaw.com/en-US/events-insights-news/bclp-and-orbital-witness-partner-on-transatlantic-generative-ai-project-to-optimize-commercial-lease-reporting.html">partnership with Bryan Cave Leighton Paisner (BCLP)</a>. It was one of many significant steps that has enabled us to bring our AI Assistant, <a href="https://www.orbitalwitness.com/copilot">Orbital Copilot</a>, to the forefront of real estate legal practice and deliver a significant impact for our customers.</div><div><br /></div><div>The genesis of Orbital Copilot dates to mid-2022, born out of our data scientists’ exploratory work with Generative AI, harnessing the power of Google’s BERT and T5 language models. The landscape suddenly shifted with OpenAI’s introduction of ChatGPT in November 2022 and the subsequent release of GPT-4. These advancements fueled our research and development (R&D) of cutting-edge internal tools, quickly proving indispensable to our legal engineers. Our presentation in June 2023, <a href="https://www.andrewthompson.co/2023/06/generative-ai-opportunities-and-risks.html">“Generative AI: Opportunities and Risks for Property Transactions,”</a> was a turning point. Our law firms immediately recognised the potential of the GenAI tools we had developed, eagerly requesting access as soon as they were available. Responding to this demand, our team rapidly advanced from a Closed Alpha of Orbital Copilot, meticulously refined with invaluable feedback from a select group of early adopters.</div><div><br /></div><div>This rapid evolution from an R&D concept to a practical tool for real estate lawyers epitomises our agility, innovation and relentless commitment to delivering genuinely valuable products to our customers. Orbital Copilot today stands as the first product of its kind in the industry: a real estate, domain-specific AI Agent that offers unparalleled analysis, review, and reporting capabilities across multiple real estate legal documents.</div><div><br /></div><div>In this blog post, we will uncover the unique aspects that make Orbital Copilot a groundbreaking product. We’ll clarify why it’s more than a rebranded ChatGPT and delve into its potential to revolutionise real estate legal practice.</div></div><div><br /></div><div><h2 style="text-align: left;">The evolution of AI in LegalTech</h2><div><br /></div><div><ul style="text-align: left;"><li><b>Classical Machine Learning (ML)</b>: In the early versions of LegalTech, NLP-based solutions primarily relied on classical supervised ML techniques. This involved collecting extensive labelled datasets and training various supervised ML models for text classification and question answering. At Orbital Witness, <a href="https://www.andrewthompson.co/2022/01/building-software-that-thinks-like.html">this was our initial approach</a>, leading to the creation of some of the industry’s most accurate ML models for classifying real estate legal text.</li></ul></div><div><ul style="text-align: left;"><li><b>Large Language Models (LLMs)</b>: The advent of LLMs in 2023 marked a paradigm shift. The traditional dependency on vast collections of labelled data for tasks like classification and question answering began to fade. Systems could be built with LLM APIs (such as OpenAI’s GPT-3.5 or Anthropic’s Claude or Google’s Gemini Pro) where they are given a portion of legal text, the “context”, along with specific instructions, the “prompt”, and the underlying LLM generates relevant responses. These systems can be advanced further by incorporating techniques such as <a href="https://medium.com/madhukarkumar/secrets-to-optimizing-rag-llm-apps-for-better-accuracy-performance-and-lower-cost-da1014127c0a">retrieval-augmented generation (RAG)</a> which enhances their ability to manage and interpret extensive context from multiple, lengthy documents. This is the most common type of system that many companies are currently building. Such a system is good for simplistic tasks but has several limitations when applied to the type of work lawyers typically do when performing due diligence. As the LegalTech AI landscape evolves, we will continue to see many more products built that summarise documents or answer specific one-off questions. The technology to perform these types of simplistic tasks is becoming well understood. The real long-term value is a dynamic AI assistant built on the AI Agent architectural pattern.</li></ul><div><br /></div></div><div><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/a/AVvXsEgcSdfeWRKh9RJUgoKvyXVAaquU2BmgL_S21zMCzBbH4YnOGdwyPRFK3hjg3CPelBZX2CZGaIQhA3RAJGbsPhI610ahONt8mhaUxNcjFQLi-j8QisCr2E2MvNTmizij9_sLcZVKf9R40SMDKT5T240Jcopdsmvl_W-9wHFNcYXHKhBQbfOs6rd6RP7GP9V5" style="margin-left: 1em; margin-right: 1em;"><img alt="" data-original-height="818" data-original-width="1488" height="352" src="https://blogger.googleusercontent.com/img/a/AVvXsEgcSdfeWRKh9RJUgoKvyXVAaquU2BmgL_S21zMCzBbH4YnOGdwyPRFK3hjg3CPelBZX2CZGaIQhA3RAJGbsPhI610ahONt8mhaUxNcjFQLi-j8QisCr2E2MvNTmizij9_sLcZVKf9R40SMDKT5T240Jcopdsmvl_W-9wHFNcYXHKhBQbfOs6rd6RP7GP9V5=w640-h352" width="640" /></a></div><br /></div><div><ul style="text-align: left;"><li><b>AI Agents</b>: Highlighted by OpenAI’s CEO, Sam Altman, at OpenAI’s DevDay in November 2023 (referenced in this <a href="https://www.ft.com/content/e628f42d-acc9-496d-be15-1ab19311735b">Financial Times article</a>), the future of AI Agents was brought into the spotlight. Altman’s announcement of customisable “GPTs” and the prospective “GPT Store” (a marketplace for ChatGPT-based chatbots) signaled a new era. While current “GPTs” are <a href="https://www.oneusefulthing.org/p/almost-an-agent-what-gpts-can-do">relatively basic AI Agents</a>, they hint at a transformative trend: AI-powered ‘agents’ capable of autonomously performing tasks and radically improving what is now possible with LLM-based applications. In LegalTech, imagine an AI Agent that, from a single query like “What is the rent for this property?”, could pinpoint relevant details across multiple documents, such as leases and deeds of variation, and then logically deduce the answer, mimicking a lawyer’s analytical process.</li></ul><div><br /></div><div><h2 style="text-align: left;">What defines an AI Agent?</h2><div><br /></div><div>In exploring the concept of AI Agents, <a href="https://lilianweng.github.io/posts/2023-06-23-agent">Lilian Weng’s insightful blog post</a> stands out, where she defines them as “LLM Powered Autonomous Agents”. Weng emphasises that the core of an AI Agent is the LLM, functioning as its ‘brain’, complemented by three critical components: Planning, Memory, and Tool Use.</div><div><br /></div><div><br /></div><div><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/a/AVvXsEjubQdtztXgQYGUj3pQz2hdX7Law1BNZa9uJhQAQTWvykfsEiVJ-5J9UqekmIszvdHvXLnybyP5dDRzjem_KDSFYtlVoS-W1QB5GpAsjmM6LcXIiF4wkVEo41xQ_ArnXZg_LxmphYFKdeNcrLtq1DLagPeSCMEReTdJT1uEdnVdNxWj17j2CyICr23YnDgu" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img alt="" data-original-height="793" data-original-width="1999" height="254" src="https://blogger.googleusercontent.com/img/a/AVvXsEjubQdtztXgQYGUj3pQz2hdX7Law1BNZa9uJhQAQTWvykfsEiVJ-5J9UqekmIszvdHvXLnybyP5dDRzjem_KDSFYtlVoS-W1QB5GpAsjmM6LcXIiF4wkVEo41xQ_ArnXZg_LxmphYFKdeNcrLtq1DLagPeSCMEReTdJT1uEdnVdNxWj17j2CyICr23YnDgu=w640-h254" width="640" /></a></div><br /><br /></div><div><br /></div><div><br /></div><div><br /></div><div><br /></div><div><br /></div><div><br /></div><div><br /></div><div><br /></div><div><br /></div><div><br /></div><div><br /></div><div><br /></div><div><br /></div><div><br /></div><div>This architecture, when expertly constructed, harbours the potential to transform the legal field. It transcends the more simplistic notion of being merely a thin wrapper around GPT-4 or a “ChatGPT for lawyers”. An AI Agent can be equipped with specialised tools designed for intricate real estate legal tasks such as looking up specific data in a land registry or determining how rent provisions might be varied by another document. The “LLM brain” skillfully determines the optimal use of these tools to execute tasks with precision and depth, as directed by a legal professional. Moreover, the AI Agent possesses the ability to reason about its generated outputs. This ability enables a real estate legal AI Agent to decide between several potential outcomes:</div><div><br /></div><div><ul style="text-align: left;"><li>Continue to delve deeper into the legal documents at it’s disposal to find a more fitting answer</li></ul><ul style="text-align: left;"><li>Request additional real estate documents or data, it feels are missing, to more comprehensively answer the question</li></ul><ul style="text-align: left;"><li>Decide it has a complete answer and present a valid response back to the lawyer who initiated the instruction</li></ul><div><br /></div><div><h2 style="text-align: left;">The Significance of AI Agents</h2><div><br /></div><div>The concept of AI Agents has been gaining substantial attention, particularly highlighted by OpenAI’s CEO, Sam Altman (see this <a href="https://www.ft.com/content/e628f42d-acc9-496d-be15-1ab19311735b">Financial Times article</a>). Altman underscores the significance of these AI agents in the overall AI landscape, with OpenAI’s upcoming GPT Store being a testament to their commitment to this platform shift. Ethan Mollick further explores this idea in his post <a href="https://www.oneusefulthing.org/p/almost-an-agent-what-gpts-can-do">Almost an Agent: What GPTs can do</a>, where he states:</div><div><br /></div><div><blockquote><i>“In their reveal of GPTs, OpenAI clearly indicated that this was just the start… GPTs can be easily integrated into with other systems, such as your email, a travel site, or corporate payment software. You can start to see the birth of true agents as a result. It is easy to design GPTs that can, for example, handle expense reports. It would have permission to look through all your credit card data and emails for likely expenses, write up a report in the right format, submit it to the appropriate authorities, and monitor your bank account to ensure payment. And you can imagine even more ambitious autonomous agents that are given a goal … and carry that out in whatever way they see fit.”</i></blockquote></div><div><br /></div><div><a href="https://a16z.com/author/olivia-moore">Olivia Moore</a>, a Consumer Partner at venture capital firm a16z, further echoed the potential of AI Agents in a <a href="https://twitter.com/omooretweets/status/1740774601876177375">recent tweet</a> at the end of 2023:</div></div></div></div><div><br /></div><div><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/a/AVvXsEjWB4lqTsk4tw0i7HUhoClNuG_eQwCHSp381jsbZwoahxfO6VqpxTdE0Jfp0cZIpXRHpdV7SpO8qFNmI6D3htU0cT76dqv5oCkyqsqgmKcjEO3oagbVkfiQXDRPfjxcTylKiYaFT1JgPbonZJrnptYhNYO7L7htRoX_JYTlTEtjCV52DKasUvL6dB5mQ2zM" style="margin-left: 1em; margin-right: 1em;"><img alt="" data-original-height="1481" data-original-width="1170" height="400" src="https://blogger.googleusercontent.com/img/a/AVvXsEjWB4lqTsk4tw0i7HUhoClNuG_eQwCHSp381jsbZwoahxfO6VqpxTdE0Jfp0cZIpXRHpdV7SpO8qFNmI6D3htU0cT76dqv5oCkyqsqgmKcjEO3oagbVkfiQXDRPfjxcTylKiYaFT1JgPbonZJrnptYhNYO7L7htRoX_JYTlTEtjCV52DKasUvL6dB5mQ2zM=w317-h400" width="317" /></a></div><div class="separator" style="clear: both; text-align: center;"><br /></div><div class="separator" style="clear: both; text-align: left;"><h2 style="clear: both; text-align: left;">The Impact of a Real Estate Legal AI Agent: Orbital Copilot</h2><div><br /></div><div class="separator" style="clear: both;"><a href="https://www.orbitalwitness.com/copilot">Orbital Copilot</a>, our innovative AI Agent at Orbital Witness, is revolutionising the way real estate legal work is conducted. It closely emulates the tasks of real estate lawyers, who often wade through extensive legal documents to perform due diligence for their clients. Here’s how Orbital Copilot is transforming the field:</div><div class="separator" style="clear: both;"><br /></div><div class="separator" style="clear: both;"><ul style="text-align: left;"><li><b>Comprehensive Document Analysis</b>: It can digest hundreds of pages of intricate legal text across numerous PDF documents whether typed, written manuscript or both</li></ul><ul style="text-align: left;"><li><b>Diverse Question Resolution</b>: From straightforward questions like “What is the date of the lease?” to more complex queries such as “How has the service charge varied?”, Orbital Copilot handles them all</li></ul><ul style="text-align: left;"><li><b>Contextual Understanding</b>: It tracks down definitions within documents to enhance understanding and reasoning</li></ul><ul style="text-align: left;"><li><b>Thorough Information Gathering</b>: Whether it’s following the breadcrumb trail across one or several documents, it ensures all necessary information is collated</li></ul><ul style="text-align: left;"><li><b>Supplementary Research</b>: It seeks out additional legal information to refine its understanding of the lawyer’s initial instructions. This could be a proprietary legal knowledge base or data from HM Land Registry</li></ul><ul style="text-align: left;"><li><b>Targeted Summarisation</b>: Orbital Copilot can summarise entire documents or specific sections across multiple documents</li></ul><ul style="text-align: left;"><li><b>Language Simplification</b>: It adeptly rephrases complex legal jargon into layman’s terms for client comprehensibility</li></ul><ul style="text-align: left;"><li><b>Trusted Referencing</b>: Orbital Copilot indicates the parts of the PDF documents it consulted, facilitating direct navigation to the primary evidence supporting its answers so lawyers can immediately see and trust where an answer came from</li></ul><ul style="text-align: left;"><li><b>Transparent Reasoning</b>: Like consulting a junior lawyer to ask how they came to their conclusion, it transparently reveals its thought process and how it arrived at specific conclusions so a lawyer can focus on the legal nature of the problem and not some “black box” technology</li></ul><div><br /></div>Specifically tailored for real estate legal tasks, Orbital Copilot’s combination of features yields remarkable time savings for lawyers. Considering the busy schedules of legal professionals, often billing in six-minute increments, Orbital Copilot’s efficiency is a game-changer. Our thorough testing with top-tier UK law firms, involving real client work, revealed that Orbital Copilot can reduce the time for a comprehensive lease report by up to 70%. Given that a single property’s lease report can take 2-10+ hours depending on complexity, this translates to substantial financial savings per property for law firms and their clients. Given the regularity of lease reports in real estate law, the cumulative efficiency and cost savings are substantial.</div></div><div><br /></div><h2 style="text-align: left;">A Glimpse into Orbital Copilot’s Functionality</h2><div><br /></div><div>Let’s take a closer look at how Orbital Copilot operates in practice. Consider this scenario where two key PDF documents are uploaded:</div><div><br /></div><div><ul style="text-align: left;"><li>Lease dated 06-06-2008</li><li>Deed of variation dated 31-03-2016</li></ul><div><br /></div></div><div>In this instance, the deed modifies several aspects of the lease, including the rent. When prompted with the query <b>“What is the rent and how has it been varied?”</b>, Orbital Copilot leaps into action. It begins by understanding the question’s context and the documents at hand. Then, it meticulously searches and reads the pertinent sections in both documents. Finally, Orbital Copilot analyses its findings, formulates a response, and presents it for review.</div><div><br /></div><div class="separator" style="clear: both; text-align: center;"><iframe allowfullscreen='allowfullscreen' webkitallowfullscreen='webkitallowfullscreen' mozallowfullscreen='mozallowfullscreen' width='484' height='268' src='https://www.blogger.com/video.g?token=AD6v5dy0CY02chD29jAxPncmoEpHZJmhkRL0KWu5RlMhO2IniNJxcjRo5HkUIRsjZwsjbMXGXneSzzV-wmfYjQBp9A' class='b-hbp-video b-uploaded' frameborder='0'></iframe></div><div><br /></div><div>Another illustration of Orbital Copilot’s capabilities is shown in the processing of a short form lease report, which includes 10 targeted questions. It’s important to note that Orbital Copilot is equipped with a variety of pre-configured lease reports, ranging from basic to highly detailed enquiries about the legal documents. The next video demonstrates the types of questions posed in the short form lease report. It also showcases how users can easily reference specific parts of the underlying documents for additional context or to validate Orbital Copilot’s responses:</div><div><br /></div><div class="separator" style="clear: both; text-align: center;"><iframe allowfullscreen='allowfullscreen' webkitallowfullscreen='webkitallowfullscreen' mozallowfullscreen='mozallowfullscreen' width='476' height='267' src='https://www.blogger.com/video.g?token=AD6v5dyAhqC_TlmzqSFcK0dcbxwlh8BHAjubH0OfFujxA70f-poCrRKviA7_nLerDI0Q9nj7g33-waODyFOJU6d2zg' class='b-hbp-video b-uploaded' frameborder='0'></iframe></div><div><br /></div><div><h2 style="text-align: left;">Engineering challenges</h2><div><br /></div><div>At Orbital Witness, leveraging the most advanced LLMs like GPT-4 is essential to meet the high standards required for legal document analysis. However, this approach presents several engineering challenges:</div><div><br /></div><div><ul style="text-align: left;"><li><b>Cost Management</b>: Utilising state-of-the-art LLMs for thorough analysis of extensive legal documents, often running into hundreds of pages, is crucial for achieving the accuracy our lawyer clients depend on. However, the use of such advanced technology incurs significant costs. Although we anticipate a decrease in expenses as Nvidia ramps up GPU production and AI labs enhance LLM efficiency, the current challenge lies in optimising our LLM usage to maintain a balance between cost-effectiveness and high-quality output.</li></ul><ul style="text-align: left;"><li><b>Resource Availability</b>: The global shortage of Nvidia GPUs, coupled with the soaring demand for LLM functionalities, has compelled LLM providers to impose caps on the number of tokens (akin to words) processed per minute through their APIs. This limitation affects our capacity to onboard new customers and influences the execution speed of tasks within Orbital Copilot. While we expect this issue to diminish as GPU availability increases and LLMs become more efficient, it remains a significant short-term constraint that requires careful management.</li></ul><ul style="text-align: left;"><li><b>Ensuring Reliability</b>: Many LLM providers, despite their technological prowess, are relatively new to managing complex, fault-tolerant services on a global scale. This inexperience can lead to occasional service fragility, manifesting as uptime issues and performance degradation. Such challenges directly impact our operations, necessitating continuous vigilance and adaptability to maintain uninterrupted service quality.</li></ul><div><br /></div></div></div><h2 style="text-align: left;">The Future of Orbital Copilot</h2><div><br /></div><div>The trajectory of Orbital Copilot is set to reach remarkable milestones. Our recent <a href="https://www.bclplaw.com/en-US/events-insights-news/bclp-and-orbital-witness-partner-on-transatlantic-generative-ai-project-to-optimize-commercial-lease-reporting.html">collaboration with Bryan Cave Leighton Paisner (BCLP)</a> is a testament to this. Through our “global design partnership,” we’re extending our lease reporting capabilities, initially honed in the UK, to BCLP’s real estate practices in both the U.K. and the U.S. This marks Orbital Witness’ ambitious leap from a UK-centric operation to a transatlantic presence in 2024. Our existing clients are already expressing eagerness to harness Orbital Copilot’s benefits on a global scale.</div><div><br /></div><div>At our core, we are a product-centric company, deeply invested in understanding and addressing our customers’ needs. This customer-first approach drives our product development, guiding us in crafting a roadmap that tackles their most pressing challenges. Currently, we are channelling our energies into developing some groundbreaking features, slated for release in the first and second quarters of 2024. These upcoming enhancements are poised to further revolutionise the landscape of real estate legal technology, strengthening Orbital Copilot’s position as a trailblazer in the field.</div><div><br /></div><h2 style="text-align: left;">Launching with an Esteemed Customer Cohort</h2><div><br /></div><div>Emerging from a highly successful private closed beta in the final quarter of 2023, we at Orbital Witness have quickly transitioned to welcoming our first batch of paying customers. This group, having been on our eagerly anticipated waitlist, represents a diverse array of prestigious companies. Their readiness to adopt Orbital Copilot’s AI Agent speaks volumes about their commitment to embracing the cutting-edge of Generative AI in real estate legal work. We are immensely proud and excited to collaborate closely with these industry leaders, each a prominent name in their respective fields:</div><div><br /></div><div><ul style="text-align: left;"><li><b>BCLP</b>: Global law firm with 31 offices worldwide and clients who represent 35% of the Fortune 500</li></ul><ul style="text-align: left;"><li><b>Clifford Chance</b>: One of the world’s largest law firms, with significant depth and range of resources across five continents</li></ul><ul style="text-align: left;"><li><b>Charles Russell Speechlys</b>: International law firm headquartered in London with offices across the UK, Europe, Asia and the Middle East</li></ul><ul style="text-align: left;"><li><b>Macfarlanes</b>: A distinctive London-based law firm with a unique combination of services built and shaped around their clients’ needs</li></ul><ul style="text-align: left;"><li><b>Ropes and Gray</b>: Global team with 13 offices on three continents and named “Law Firm of the Year” by The American Lawyer in 2022 and ranked number one on The American Lawyer’s A-List of elite firms</li></ul><ul style="text-align: left;"><li><b>Walker Morris</b>: Independent law firm with a first-class international reputation</li></ul><ul style="text-align: left;"><li><b>Thomson Snell and Passmore</b>: The oldest law firm in operation tracing back to the late 16th century</li></ul><ul style="text-align: left;"><li><b>Thompson, Smith and Puxon</b>: Established in 1879, TSP has grown to be one of the leading law firms in Essex</li></ul><ul style="text-align: left;"><li><b>Able UK</b>: Market leader in wind energy & marine decommissioning along with being a significant land developer and port & vehicle storage operator</li></ul><div><br /></div></div></div></div></div><div><h2 style="text-align: left;">Customer Testimonials</h2><div><br /></div><div><blockquote><i>“Orbital Copilot is next generation legal technology and is helping us continue to focus our Real Estate lawyers’ time on the areas that are most valuable to clients. Our IT strategy has always been to seek out the best technology tools for our needs and our collaboration with Orbital Witness is a key plank of our Real Estate AI plan.”</i></blockquote></div><div><br /></div><div><span> </span><span> </span><b>– Matt Taylor, Partner @ Clifford Chance</b></div><div><br /></div><div></div><blockquote><div><i>“BCLP was an early adopter of Orbital Witness’ products, and we are thrilled to collaborate with the company on an AI solution that will provide marked benefits to our clients by accelerating lease reporting and enhancing real estate due diligence. We’re also excited to see so many of our lawyers deeply engaged in the development of leading-edge technology and experiencing the potential of generative AI firsthand.”</i></div><div></div></blockquote><div><br /></div><div> <b> </b><b>– Samant Narula, BCLP’s U.K. Head of Real Estate</b></div><div><br /></div><div><blockquote><i>“The potential of Orbital Copilot is stunning. A time saving AI tool which will increase our efficiency. Icing on the cake of Orbital Witness’ services.”</i></blockquote></div><div><br /></div><div><span> </span><span> </span><b>– Clive Gotley, Head of Legal @ Gridserve</b></div><div><br /></div><div><blockquote><i>“Copilot is the kind of tool that every lawyer wishes they had. It enables you to review documents quickly and efficiently, whilst also allowing you to check and verify the information, ensuring that the end result is the one that you want.”</i></blockquote></div><div><br /></div><div><span> </span><span> </span><b>– Amy Shuttleworth, Associate @ Charles Russell Speechlys</b></div></div><div><br /></div><div><h2 style="text-align: left;">Conclusion: The Future is Here with AI Agents</h2><div><br /></div><div><ul style="text-align: left;"><li><b>AI Agents - A LegalTech Revolution</b>: The future of LegalTech is being reshaped by AI Agents. Their advanced capabilities and adaptability make them indispensable tools to begin integrating into modern legal practices</li></ul><ul style="text-align: left;"><li><b>Orbital Copilot - Leading the Charge</b>: Orbital Witness has pioneered the development of the world’s premier AI Agent tailored for real estate legal work. Orbital Copilot is not just an innovation; it’s a game-changer, already enhancing due diligence and reporting processes by an impressive 70%</li></ul><ul style="text-align: left;"><li><b>Customer Acclaim</b>: The response from our customers has been overwhelmingly positive. The efficiency and precision Orbital Copilot brings to their client work have made it an essential component of their legal toolkit</li></ul><ul style="text-align: left;"><li><b>Join the Early Adopters</b>: For those eager to be part of this transformative journey, we have a limited number of spots in our <a href="https://www.orbitalwitness.com/copilot">early adopters cohort</a>. Interest has been high, so we encourage you to sign up quickly to secure your place as more slots become available</li></ul></div><div><br /></div><div>As we move forward, Orbital Copilot continues to set new benchmarks in the realm of real estate legal technology. Stay tuned for more updates and innovations as we navigate this exciting frontier.</div></div><p></p>Andrewhttp://www.blogger.com/profile/03252749023463514361noreply@blogger.com0tag:blogger.com,1999:blog-1847717784907730054.post-27207740810304118692023-11-01T21:53:00.009+00:002023-11-02T00:10:48.778+00:00My team has officially shipped something very exciting today!<p></p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/a/AVvXsEi4nhem1F8uguIn5UtvAypbkMKXnNoqSdDBeMP2_U-vR9IfMLBadbGVt8qNy3RVri3GKkMu1lVE2DeQ2LgkmcKKC_x0SUvVfC2gIJALWGGEWKwTZlHwdfCnJfHFTysCG_OAxk3YSGjEsXvxm6hENe4XbZIFfJxfF2z7Q7vPlzf3hPUoduRJdiviq-5puIt7" style="margin-left: 1em; margin-right: 1em;"><img alt="" data-original-height="1108" data-original-width="2216" height="200" src="https://blogger.googleusercontent.com/img/a/AVvXsEi4nhem1F8uguIn5UtvAypbkMKXnNoqSdDBeMP2_U-vR9IfMLBadbGVt8qNy3RVri3GKkMu1lVE2DeQ2LgkmcKKC_x0SUvVfC2gIJALWGGEWKwTZlHwdfCnJfHFTysCG_OAxk3YSGjEsXvxm6hENe4XbZIFfJxfF2z7Q7vPlzf3hPUoduRJdiviq-5puIt7=w400-h200" width="400" /></a></div><p>Back in <a href="https://www.andrewthompson.co/2023/06/generative-ai-opportunities-and-risks.html">June 2023 I gave a presentation</a> titled "Generative AI: Opportunities and risks for property transactions" in which I demoed an internal AI Assistant project that my team and I had been building ever since ChatGPT launched to the world in November 2022 and it became abundantly obvious this new Generative AI technology had the potential to transform the legal industry. We received a tremendous amount of engagement off the back of my presentation and much of it was from customers wanting to get their hands on this AI Assistant that we built so they could try it out for themselves. Because our initial AI Assistant was only an internal tool, we need to pull it out of our backend application and productionise it for law firms and other companies so they could begin trialing it and giving us feedback. </p><p>Fast forward a few months once we had ironed out a number of challenging technical problems with building a bleeding edge AI system built on top of the latest large language models (LLMs), our own customised prompt engineering, a retrieval-augmented generation (RAG) system and our own proprietary legal specific document OCR and structuring technology all the while controlling for the eye-watering costs of some of the more powerful LLMs like OpenAI's GPT-4. </p><p>Today <a href="https://www.bclplaw.com/en-US/events-insights-news/bclp-and-orbital-witness-partner-on-transatlantic-generative-ai-project-to-optimize-commercial-lease-reporting.html">BCLP and Orbital Witness have publicly released the details</a> of our <i>"global design partnership to test, optimize and deploy Orbital Witness’ lease reporting capabilities in its latest generative artificial intelligence (AI) solution, Orbital Copilot, across BCLP’s U.K. and U.S. real estate practices." </i>I'm proud to say that the AI Assistant I showcased at my presentation back in June 2023 has now taken on a life of it's own as Orbital Copilot and has a ton more functionality, improved accuracy, ability to work with any legal documents and is initially focused on lease reporting for lawyers and legal professionals as we build this out for BCLP and others. The press release goes on to say: <i>"Orbital Copilot is the first real estate-sector-specific AI assistant created for real estate lawyers that is capable of analyzing and reporting on any real estate document. By leveraging Orbital Witness’ market-leading capabilities in real estate-specific AI and the deep sector knowledge and experience of BCLP’s global real estate practice, this first-in-market collaboration seeks to use this cutting-edge technology to enhance BLCP’s reporting on complex commercial leases, providing accelerated insights to their clients."</i></p><p>I'm also quoted in the press release alongside <a href="https://www.linkedin.com/in/samant-narula-4587619/">Samant Narula</a>, BCLP’s U.K. head of real estate: </p><p><i></i></p><blockquote><i>Andrew Thompson, chief technology officer at Orbital Witness, says, “A key part of our strategy with Orbital Copilot is to develop real estate-domain-specific AI that enables us to deliver solutions for property professionals that are incomparable in terms of capability, quality and reliability. This collaboration and pilot program with BCLP allows us to demonstrate and test that value at a firm that shares our ambitions and vision for this technology.”</i></blockquote><p></p><p>BCLP has one of the largest legal real estate practises in the country, so they're a brilliant law firm to partner with for the ground breaking Generative AI product called Orbital Copilot that we've just built and launched to the world.</p><p><br /></p>Andrewhttp://www.blogger.com/profile/03252749023463514361noreply@blogger.com0tag:blogger.com,1999:blog-1847717784907730054.post-46598492191763910992023-06-21T21:07:00.001+01:002023-06-21T21:08:04.148+01:00Generative AI: Opportunities and risks for property transactions<p>On June 14, 2023, <a href="https://www.eventbrite.com/e/generative-ai-opportunities-and-risks-for-property-transactions-tickets-634232836487">Orbital Witness held an event</a> for #LondonTechWeek to explore how Generative AI will revolutionise property due diligence. I gave the keynote presentation to a room of partners and innovation specialists from commercial law firms, residential conveyancing firms and LegalTech companies. </p><p>Here is the full 38min video of that presentation along a product demo at the 32min mark. The product I demoed is an internal tool my team of data scientists and legal engineers built using a GPT-4 based agent and vector search on top of legal documents from <a href="https://www.gov.uk/government/organisations/land-registry">HMLR</a> such as title registers and scanned PDFs of leases and deeds of variation:</p><p><b><iframe allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share" allowfullscreen="" frameborder="0" height="360" src="https://www.youtube.com/embed/80t9m3cv-h4" title="YouTube video player" width="640"></iframe></b></p><p>Here is the complete slide deck I presented for my keynote with all transitions intact (PDF version at the end of this blog post):</p>
<iframe allowfullscreen="true" frameborder="0" height="389" mozallowfullscreen="true" src="https://docs.google.com/presentation/d/e/2PACX-1vSKczlamLpK_1MzSYOCWuVvK_4PFigjLbJtNYQsvcIsZCyNjfPJjNr2dYsjNG1_x8TYctQQ_WpoZJff/embed?start=false&loop=true&delayms=3000" webkitallowfullscreen="true" width="640"></iframe><p><span style="text-align: justify;">Here are all the references associated with various slides:</span></p><div style="text-align: justify;"><ul><li style="text-align: left;">Slide 4: <a href="https://www.mckinsey.com/featured-insights/mckinsey-explainers/what-is-ai">https://www.mckinsey.com/featured-insights/mckinsey-explainers/what-is-ai</a></li><li style="text-align: left;">Slide 5: <a href="https://a16z.com/2023/06/06/ai-will-save-the-world">https://a16z.com/2023/06/06/ai-will-save-the-world</a> </li><li style="text-align: left;">Slide 6: <a href="https://www.cliffordchance.com/insights/thought_leadership/ai-and-tech/generative-ai-the-big-questions.html">https://www.cliffordchance.com/insights/thought_leadership/ai-and-tech/generative-ai-the-big-questions.html</a> </li><li><div style="text-align: left;">Slide 9: <a href="https://openai.com/gpt-4">https://openai.com/gpt-4</a> </div></li><li style="text-align: left;">Slide 10/11: <a href="https://ourworldindata.org/brief-history-of-ai">https://ourworldindata.org/brief-history-of-ai</a> </li><li style="text-align: left;">Slide 12: <a href="https://arxiv.org/abs/1706.03762">https://arxiv.org/abs/1706.03762</a></li><li style="text-align: left;">Slide 14: <a href="https://uk.news.yahoo.com/chatgpt-on-track-to-surpass-100-million-users-faster-than-tiktok-or-instagram-ubs-214423357.html">https://uk.news.yahoo.com/chatgpt-on-track-to-surpass-100-million-users-faster-than-tiktok-or-instagram-ubs-214423357.html</a></li><li style="text-align: left;">Slide 16/17: <a href="https://writings.stephenwolfram.com/2023/02/what-is-chatgpt-doing-and-why-does-it-work">https://writings.stephenwolfram.com/2023/02/what-is-chatgpt-doing-and-why-does-it-work</a></li><li style="text-align: left;">Slide 18: <a href="https://ourworldindata.org/brief-history-of-ai">https://ourworldindata.org/brief-history-of-ai</a></li><li style="text-align: left;">Slide 19: <a href="https://twitter.com/bressane/status/1664939696907243520">https://twitter.com/bressane/status/1664939696907243520</a></li><li style="text-align: left;">Slide 30/31: <a href="https://www.anthropic.com/index/100k-context-windows">https://www.anthropic.com/index/100k-context-windows</a></li><li style="text-align: left;">Slide 33: <a href="https://www.iit.edu/news/gpt-4-passes-bar-exam">https://www.iit.edu/news/gpt-4-passes-bar-exam</a></li><li style="text-align: left;">Slide 34/35/36: <a href="https://papers.ssrn.com/sol3/papers.cfm?abstract_id=4389233">https://papers.ssrn.com/sol3/papers.cfm?abstract_id=4389233</a></li><li style="text-align: left;">Slide 39: <a href="https://www.bbc.co.uk/news/business-57914323">https://www.bbc.co.uk/news/business-57914323</a></li><li style="text-align: left;">Slide 42: <a href="https://www.bbc.co.uk/news/world-us-canada-65735769">https://www.bbc.co.uk/news/world-us-canada-65735769</a></li><li style="text-align: left;">Slide 45: <a href="https://arxiv.org/pdf/2201.11903.pdf">https://arxiv.org/pdf/2201.11903.pdf</a></li><li style="text-align: left;">Slide 52: <a href="https://fortune.com/2023/05/19/chatgpt-banned-workplace-apple-goldman-risk-privacy">https://fortune.com/2023/05/19/chatgpt-banned-workplace-apple-goldman-risk-privacy</a></li><li style="text-align: left;">Slide 54: <a href="https://openai.com/policies/api-data-usage-policies">https://openai.com/policies/api-data-usage-policies</a></li><li style="text-align: left;">Slide 55: <a href="https://azure.microsoft.com/en-gb/products/cognitive-services/openai-service#faq">https://azure.microsoft.com/en-gb/products/cognitive-services/openai-service#faq</a></li><li style="text-align: left;">Slide 57: <a href="https://www.ben-evans.com/benedictevans/2020/5/16/not-even-wrong">https://www.ben-evans.com/benedictevans/2020/5/16/not-even-wrong</a></li><li style="text-align: left;">Slide 61: <a href="https://www.bbc.co.uk/news/technology-44351214">https://www.bbc.co.uk/news/technology-44351214</a></li><li style="text-align: left;">Slide 62: <a href="https://en.wikipedia.org/wiki/GitHub_Copilot">https://en.wikipedia.org/wiki/GitHub_Copilot</a></li><li style="text-align: left;">Slide 63: <a href="https://github.blog/2023-03-22-github-copilot-x-the-ai-powered-developer-experience">https://github.blog/2023-03-22-github-copilot-x-the-ai-powered-developer-experience</a></li><li style="text-align: left;">Slide 64/65: <a href="https://github.blog/2023-03-22-github-copilot-x-the-ai-powered-developer-experience">https://github.blog/2023-03-22-github-copilot-x-the-ai-powered-developer-experience</a> and <a href="https://github.blog/2022-09-07-research-quantifying-github-copilots-impact-on-developer-productivity-and-happiness">https://github.blog/2022-09-07-research-quantifying-github-copilots-impact-on-developer-productivity-and-happiness</a></li></ul><div style="text-align: left;"><div><hr /></div><div style="text-align: justify;"><div class="separator" style="clear: both; text-align: left;"><br /></div><div class="separator" style="clear: both; text-align: left;"><div class="separator" style="clear: both; text-align: center;"></div></div></div></div><div style="text-align: left;"></div><div style="text-align: left;"><iframe height="820" src="https://drive.google.com/file/d/1JMfj7Sel1jEdhemcCLJtPd_CTPD8Fmvu/preview" width="660"></iframe></div></div>Andrewhttp://www.blogger.com/profile/03252749023463514361noreply@blogger.com0tag:blogger.com,1999:blog-1847717784907730054.post-71782846099637112442023-03-09T22:44:00.065+00:002023-07-17T15:38:20.393+01:00A collection of curated learning materials about Generative AI and Large Language Models (LLMs) such as GPT-4<div class="separator" style="clear: both; text-align: left;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjVBzEtE4GoZ-A9lBSj2bF5IeBmlDr37JlDL6n2wgit3iAL7o7ddUHCOHvNTAMwg-JqN5KL8ftUW5kC7LFzaM2RDWvDdSM7QHJT8IIjlRUkoSAejXh3M1e3SCizIPqKz4cWOwSuSCb-mVu3K4U0WRRzENUx17_K4P1iBxdvVY4iHwW3-D94goTkyMraP2C7/s2000/2657-Emerging-LLM-App-Stack-R2-1-of-4-2.webp" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" data-original-height="1400" data-original-width="2000" height="448" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjVBzEtE4GoZ-A9lBSj2bF5IeBmlDr37JlDL6n2wgit3iAL7o7ddUHCOHvNTAMwg-JqN5KL8ftUW5kC7LFzaM2RDWvDdSM7QHJT8IIjlRUkoSAejXh3M1e3SCizIPqKz4cWOwSuSCb-mVu3K4U0WRRzENUx17_K4P1iBxdvVY4iHwW3-D94goTkyMraP2C7/w640-h448/2657-Emerging-LLM-App-Stack-R2-1-of-4-2.webp" width="640" /></a></div><p><br /></p><p><br /></p><p><br /></p><p><br /></p><p><br /></p><p><br /></p><p><br /></p><p><br /></p><p><br /></p><p><br /></p><p><br /></p><p><br /></p><p><br /></p><p><br /></p><p>Since <a href="https://openai.com/blog/chatgpt">ChatGPT came onto the scene on 30th November, 2022</a>, I've been doing a deep dive finding out what exactly LLMs and specifically ChatGPT are capable of, what their limitations are and how we can use them at <a href="https://www.orbitalwitness.com/">Orbital Witness</a> (the company I'm CTO at). This was obviously important since we're currently <a href="https://www.andrewthompson.co/2022/01/building-software-that-thinks-like.html">building software that 'thinks' like a lawyer</a> using the latest natural language processing (NLP). </p><p>Given my current ongoing research on the topic, I'm going to use this blog post as a curated collection of the best content I've read across a variety of themes related to LLMs / ChatGPT / Dall-E 2 / etc. I'll update it regularly with links to new blog posts, podcasts, videos and anything else I find particularly compelling. The date in parentheses is when I added the link to this blog post and over time it will provide a change-log for easier reading. </p><h2 style="text-align: left;"><b>Overview</b></h2><p></p><ul style="text-align: left;"><li><a href="https://cobusgreyling.medium.com/the-large-language-model-landscape-9da7ee17710b">The Large Language Model Landscape</a> by Cobus Greyling (Nov '22 - Pre-ChatGPT)</li><li><a href="https://jmcdonnell.substack.com/p/the-near-future-of-ai-is-action-driven">The Near Future of AI is Action-Driven</a> by John McDonnell (Nov '22 - Pre-ChatGPT)</li><li><a href="https://sifted.eu/articles/what-is-generative-ai">Why generative AI is getting VCs all excited</a> by Tim Smith (Jan '23)</li><li><a href="https://writings.stephenwolfram.com/2023/02/what-is-chatgpt-doing-and-why-does-it-work">"What Is ChatGPT Doing … and Why Does It Work?"</a> by Stephen Wolfram (Feb '23)</li><ul><li>Stephen also has a <a href="https://www.youtube.com/watch?v=flXrLGPY3SU">3hr+ Youtube video by the same name</a></li></ul><li><a href="https://www.version1.com/an-analysis-of-chatgpt-and-openai-gpt3-how-to-use-it-for-your-business">An Analysis of ChatGPT and OpenAI GPT-3: How to Use it For Your Business</a> by Version 1 (Feb '23)</li><li><a href="https://a16z.com/2023/01/19/who-owns-the-generative-ai-platform/">Who Owns the Generative AI Platform?</a> by Matt Bornstein, Guido Appenzeller, and Martin Casado of a16z (Feb '23)</li><li><a href="https://www.impromptubook.com/wp-content/uploads/2023/03/impromptu-rh.pdf">Impromptu: Amplifying Our Humanity Through AI</a> by Reid Hoffman with GPT-4 (Mar '23)</li><li><a href="https://techcrunch.com/2023/03/23/chatgpt-everything-you-need-to-know-about-the-ai-powered-chatbot/">ChatGPT: Everything you need to know about the AI-powered chatbot</a> by TechCrunch (Mar '23)</li><li><a href="https://twitter.com/aakashg0/status/1639674879359483906?s=20">HUGE week in generative AI</a> by Aakash Gupta (Mar '23)</li><li><a href="https://www.youtube.com/watch?v=L_Guz73e6fw">Sam Altman: OpenAI CEO on GPT-4, ChatGPT, and the Future of AI</a> by Lex Fridman Podcast (Mar '23)</li><li><a href="https://github.com/sw-yx/ai-notes">AI Notes</a> by Latent Space Newsletter (Apr '23)</li><li><a href="https://arxiv.org/pdf/2210.03629.pdf">ReAct: Synergizing Reasoning and Acting in Language Models</a> (Apr '23)</li><li><a href="https://arxiv.org/abs/2304.13712">Harnessing the Power of LLMs in Practice: A Survey on ChatGPT and Beyond</a> (Apr '23)</li><li><a href="https://www.youtube.com/watch?v=ax-uKawA-c8">Building Defensible Products with LLMs</a> by Raza Habib (May '23)</li><li><a href="https://web.archive.org/web/20230531203946/https://humanloop.com/blog/openai-plans">OpenAI's plans according to Sam Altman</a> by Raza Habib (Jun '23)</li><li><a href="https://a16z.com/2023/06/20/emerging-architectures-for-llm-applications/">Emerging Architectures for LLM Applications</a> by 16z (Jun '23)</li><li><a href="https://greylock.com/greymatter/the-new-new-moats/">The New New Moats - Why Systems of Intelligence are still the next defensible business model</a> by Jerry Chen (Jun '23)</li><li><a href="https://www.sequoiacap.com/article/llm-stack-perspective/">The New Language Model Stack</a> by Michelle Fradin and Lauren Reeder (Jun '23)</li></ul><p></p><h2 style="text-align: left;"><b>Prompt Engineering</b></h2><p></p><ul style="text-align: left;"><li><a href="https://www.amazon.science/blog/emnlp-prompt-engineering-is-the-new-feature-engineering">EMNLP: Prompt engineering is the new feature engineering</a> by Larry Hardesty (Mar '23)</li><li><a href="https://dallery.gallery/the-dalle-2-prompt-book/">The DALL·E 2 Prompt Book</a> by Guy Parsons (Mar '23)</li><ul><li>a16z's podcast with Guy Parsons titled <a href="https://podcasts.apple.com/us/podcast/unlocking-creativity-with-prompt-engineering/id842818711">Unlocking Creativity with Prompt Engineering</a></li></ul><li><a href="https://learnprompting.org/">Learn Prompting Course</a> (Mar '23)</li><li><a href="https://minimaxir.com/2023/03/new-chatgpt-overlord/">ChatGPT's API is So Good and Cheap, It Makes Most Text Generating AI Obsolete</a> by Max Woolf (Mar '23)</li><li><a href="https://twitter.com/nickfloats/status/1635116672054079488?s=20">Using ChatGPT with Additive Prompting for Midjourney Image Creation</a> by Nick St. Pierre (Mar '23)</li><li><a href="https://twitter.com/danberridge/status/1635469658408222720?s=20">Examples of Bing Chat Prompt Engineering</a> by Dan Berridge (Mar '23)</li><li><a href="https://twitter.com/heybarsee/status/1640368020949221376">ChatGPT prompts to finish hours of work in seconds</a> by Barsee (Mar '23)</li><li><a href="https://lilianweng.github.io/posts/2023-03-15-prompt-engineering/">Prompt Engineering</a> by Lilian Weng (Mar '23)</li><li><a href="https://github.com/openai/openai-cookbook/blob/834181d5739740eb8380096dac7056c925578d9a/techniques_to_improve_reliability.md">Techniques to improve reliability</a> by OpenAI Cookbook (Apr '23)</li><li><a href="https://mitchellh.com/writing/prompt-engineering-vs-blind-prompting">Prompt Engineering vs. Blind Prompting</a> by Mitchell Hashimoto (Apr '23)</li><li><a href="https://mlops.community/fine-tuning-vs-prompt-engineering-llms">Fine Tuning vs Prompt Engineering Large Language Models</a> by Niels Bantilan (May '23)</li></ul><p></p><h2 style="text-align: left;"><b>Managing Context</b></h2><div><b><div class="separator" style="clear: both; text-align: center;"><br /></div><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/a/AVvXsEgElwyucJTddrX38C-Q6yYEcap9vPEZqqBXUbONJ6pWX7zIn84rmnqMvFbCQX1pVUnNJHQSP8GoRGV0nemPR3qCgrtDWTzORycOqc7MmlBXBVHJVy3wycJX84hkOuSi1gFqxu0FAJrt0sxptsn_HwAd9ICY3fks0CteZq0J9el0FO_MVDqDVH_R7g4igg" style="margin-left: 1em; margin-right: 1em;"><img alt="" data-original-height="1080" data-original-width="1920" height="225" src="https://blogger.googleusercontent.com/img/a/AVvXsEgElwyucJTddrX38C-Q6yYEcap9vPEZqqBXUbONJ6pWX7zIn84rmnqMvFbCQX1pVUnNJHQSP8GoRGV0nemPR3qCgrtDWTzORycOqc7MmlBXBVHJVy3wycJX84hkOuSi1gFqxu0FAJrt0sxptsn_HwAd9ICY3fks0CteZq0J9el0FO_MVDqDVH_R7g4igg=w400-h225" width="400" /></a></div></b></div><p></p><ul style="text-align: left;"><li><a href="https://gpt-index.readthedocs.io/en/latest/index.html">LlamaIndex 🦙 (GPT Index)</a> (Mar '23)</li><li><a href="https://lifearchitect.ai/gpt-4/">Collection of content on the upcoming GPT4</a> by Life Architect (Mar '23)</li><li><a href="https://medium.com/@boredgeeksociety/openai-released-gpt4-api-everything-you-need-to-know-5d2520b73f4c">OpenAI released GPT4-API: everything you need to know!</a> by Wassim Jouini (Mar '23)</li><ul><li><i>"Larger context means more complex applications: A 32k tokens context means about 25 000 words or 50 pages! This is huge! It means it can process full reports, short books or long chapters, taxation code, and so on, enabling scenarios that were impossible with ChatGPT’s 4k token limitation!"</i></li></ul><li><a href="https://twitter.com/ericciarla/status/1641527314172223488">Hybrid Search Idea</a> by Eric Ciarla (Apr '23)</li><li><a href="https://docs.langchain.com/docs/components/chains/index_related_chains">Index-related Chains</a> by LangChain (Apr '23)</li><li><a href="https://www.anthropic.com/index/100k-context-windows">Introducing 100K Context Windows</a> by Anthropic (Mat '23)</li></ul><h2 style="text-align: left;"><b>Accuracy</b></h2><div><ul style="text-align: left;"><li><a href="https://github.com/mjbommar/gpt-takes-the-bar-exam">GPT Takes the Bar Exam</a> by Michael Bommarito (Mar '23)</li><li><a href="https://arxiv.org/pdf/2109.07958.pdf">TruthfulQA: Measuring How Models Mimic Human Falsehoods</a> by Stephanie Lin, Jacob Hilton and Owain Evans (Mar '23)</li><li><a href="https://jfan001.medium.com/how-we-cut-the-rate-of-gpt-hallucinations-from-20-to-less-than-2-f3bfcc10e4ec">How we cut the rate of GPT hallucinations from 20%+ to less than 2%</a> by Jason Fan (Mar '23)</li><li><a href="https://gptleetcode.com/">The results of OpenAI's ChatGPT ran against every (non-premium) Leetcode problem</a> (Mar '23)</li><li><a href="https://openai.com/research/gpt-4">GPT-4 Research Blog Post</a> by OpenAI (Mar '23)</li></ul><h2 style="text-align: left;">Hallucinating</h2></div><div><ul style="text-align: left;"><li><a href="https://www.danieldjohnson.com/2023/03/27/uncertain_simulators/">Uncertain Simulators Don't Always Simulate Uncertain Agents</a> by Daniel D. Johnson (Mar '23)</li></ul></div><p></p><h2 style="text-align: left;"><b>Productivity</b></h2><p></p><a href="https://oneusefulthing.substack.com/p/secret-cyborgs-the-present-disruption"></a><ol style="text-align: left;"><a href="https://oneusefulthing.substack.com/p/secret-cyborgs-the-present-disruption"></a></ol><ul style="text-align: left;"><li><a href="https://oneusefulthing.substack.com/p/secret-cyborgs-the-present-disruption"></a><a href="https://oneusefulthing.substack.com/p/secret-cyborgs-the-present-disruption">"Secret Cyborgs: The Present Disruption in Three Papers"</a> by Ethan Mollick (Mar '23)</li><ul><li>Ethan's Twitter (he's a Professor at Wharton studying innovation & startups) has been one of the best resources to see what kinds of things are possible with ChatGPT</li><ul><li>Tweet: <a href="https://twitter.com/emollick/status/1632919984916553731?s=20">Turn ChatGPT into a (pretty good) adaptive tutor</a></li><li>Tweet: <a href="https://twitter.com/emollick/status/1633175746435862529">Here's another terrific use case for ChatGPT for any team building products: have it do a premortem, telling you stories about how you could fail.</a></li></ul></ul><li><a href="https://tomtunguz.com/llm-impact-gdp/">Which Increases Productivity More : The Advent of Personal Computer or a Large-Language Model?</a> by Tomasz Tunguz</li></ul><ol style="text-align: left;"></ol><h2 style="text-align: left;"><b>User Experience (UX)</b></h2><div><ul style="text-align: left;"><li><a href="https://www.youtube.com/watch?v=fayBEXSKyoI">Say Hi To ChatSpot.ai: The All-In-One A.I. Powered Chat App For Growing Better</a> by Dharmesh Shah, CTO at HubSpot (Mar '23)</li><li><a href="https://twitter.com/danberridge/status/1633604748132909058">Here are 7 large companies that now incorporate ChatGPT’s model in their product</a> (Hubspot, Brex, Slack, Snapchat, Shopify, Instacart, Quizlet) by Dan Berridge (Mar '23)</li><li><a href="https://twitter.com/gdb/status/1638949234681712643?s=12&t=q7hud9JI0Ur0pdMNoJhAZw">ChatGPT Plugins</a> by Greg Brockman (Mar '23)</li></ul><h2 style="text-align: left;"><b>Traction of LLMs</b></h2></div><div><ul style="text-align: left;"><li><a href="https://twitter.com/YahooFinance/status/1621814843475988480?s=20">ChatGPT on track to surpass 100 million users faster than TikTok or Instagram: UBS</a> by Yahoo Finance (Feb '23)</li><li><a href="https://tomtunguz.com/user-acquisition-curiosity">ChatGPT & Taylor Swift</a> by Tomasz Tunguz (Apr '23)</li></ul><h2 style="text-align: left;"><b>Long-term Costs</b></h2></div><div><ul style="text-align: left;"><li><a href="https://twitter.com/labenz/status/1630284912853917697">OpenAI's leaked Foundry pricing says a lot</a> by Nathan Labenz (Feb '23)</li><li><a href="https://arstechnica.com/information-technology/2023/03/you-can-now-run-a-gpt-3-level-ai-model-on-your-laptop-phone-and-raspberry-pi">You can now run a GPT-3-level AI model on your laptop, phone, and Raspberry Pi</a> by ArsTechnica (Mar '23)</li><li><a href="https://help.openai.com/en/articles/7127956-how-much-does-gpt-4-cost">How much does GPT-4 cost? (gpt-4 & gpt-4-32k)</a> by OpenAI (Mar '23)</li></ul><h2 style="text-align: left;">How-to</h2></div><div><ul style="text-align: left;"><li><a href="https://www.lennysnewsletter.com/p/i-built-a-lenny-chatbot-using-gpt">I built a Lenny chatbot using GPT-3. Here’s how to build your own</a> by Dan Shipper</li><li><a href="https://github.com/microsoft/visual-chatgpt">Visual ChatGPT</a> by Microsoft (Mar '23 prior to GPT-4's release)</li><li><a href="https://www.youtube.com/watch?v=outcGtbnMuQ">GPT-4 Developer Livestream (Recorded)</a> by OpenAI (Mar '23)</li><li><a href="https://twitter.com/rungreatclasses/status/1641793171423219713">Challenging yourself with GPT panels </a>by Jeremy Nguyen (Apr '23)</li><li><a href="https://twitter.com/rasbt/status/1642880757566676992">BloombergGPT</a> by Sebastian Raschka (Apr '23)</li><li><a href="https://www.databricks.com/blog/2023/04/12/dolly-first-open-commercially-viable-instruction-tuned-llm">Free Dolly: Introducing the World's First Truly Open Instruction-Tuned LLM</a> by databricks (Apr '23)</li><li><a href="https://huyenchip.com/2023/04/11/llm-engineering.html">Building LLM applications for production</a> by Chip Huyen (Apr '23)</li><li><a href="https://www.youtube.com/watch?v=rpjLTHrl-S4">Large Language Models in Production Round-table Conversation</a> (57min video) by MLOps.community (Apr '23)</li><li><a href="https://enchanting-trader-463.notion.site/Midjourney-AI-Guide-41eca43809dd4d8fa676e648436fc29c">Midjourney AI Guide</a> by Barsee (Apr '23)</li></ul><div><br /></div><div><div class="separator" style="clear: both; text-align: center;"><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/a/AVvXsEhdDris266Z3rNMqAVCwVw90qNKp0Ev3tHZHT0VbCLWutVVVUmoZeFVNjkIHG242zm_iTfc1OHXMr7HLue-p4E8yjG1EkjE-8p4tVbOND5xBnCUc0kxmqRJUesI94M3VXneLyGK_Rs0L-Ood6h2ikav4iZP4B7bbhQ1Ttgm6Mzrjhd4nyx3Zb9u6a22Ow" style="margin-left: 1em; margin-right: 1em;"><img alt="" data-original-height="975" data-original-width="1319" height="474" src="https://blogger.googleusercontent.com/img/a/AVvXsEhdDris266Z3rNMqAVCwVw90qNKp0Ev3tHZHT0VbCLWutVVVUmoZeFVNjkIHG242zm_iTfc1OHXMr7HLue-p4E8yjG1EkjE-8p4tVbOND5xBnCUc0kxmqRJUesI94M3VXneLyGK_Rs0L-Ood6h2ikav4iZP4B7bbhQ1Ttgm6Mzrjhd4nyx3Zb9u6a22Ow=w640-h474" width="640" /></a></div><br /><br /></div><br /><br /></div></div><p></p>Andrewhttp://www.blogger.com/profile/03252749023463514361noreply@blogger.com0tag:blogger.com,1999:blog-1847717784907730054.post-53107061875311553402022-04-27T00:16:00.006+01:002023-06-21T21:19:13.125+01:00Links To Previous Posts<div dir="ltr" style="text-align: left;" trbidi="on">
<h2 style="text-align: left;">Team</h2>
<ul><li><a href="https://www.andrewthompson.co/2022/04/flexible-working-at-orbital-witness.html">Flexible working at Orbital Witness</a> (2022)</li>
<li><a href="https://www.andrewthompson.co/2020/04/simple-template-for-tracking-okrs.html">Simple Template For Tracking OKRs Weekly And Quarterly</a> (2020)</li>
<li><a href="https://www.andrewthompson.co/2019/01/how-we-hire-product-developers-at.html">How We Hire Product Developers at Appear Here</a> (2019)</li>
<li><a href="https://www.andrewthompson.co/2019/01/appear-heres-case-study-with-hired.html">Appear Here's Case Study with Hired</a> (2019)</li>
<li><a href="https://www.andrewthompson.co/2017/06/my-podcast-with-aws-startup-stories.html">My podcast with AWS Startup Stories about hiring for startups</a> (2017)</li>
<li><a href="http://www.andrewthompson.co/2015/07/10-steps-to-priming-kanban.html">10 Steps To Priming Kanban</a> (2015)</li>
<li><a href="http://www.andrewthompson.co/2013/09/building-product-roadmaps-with-kanban.html">Building Product Roadmaps with Kanban</a> (2013)</li>
<li><a href="http://www.andrewthompson.co/2013/08/3-responsibilities-of-product-management.html">3 Responsibilities of Product Management</a> (2013)</li>
<li><a href="http://www.andrewthompson.co/2011/03/continuous-integration-tooling.html">Continuous Integration: Tooling, Processes & Implicit Communication</a> (2011)</li>
</ul><div><br /></div>
<h2 style="text-align: left;">
Technical</h2>
<ul style="text-align: left;">
<li><a href="https://www.andrewthompson.co/2023/06/generative-ai-opportunities-and-risks.html">Generative AI: Opportunities and risks for property transactions</a> (2023)</li><li><a href="https://www.andrewthompson.co/2023/03/a-collection-of-curated-learning.html">A collection of curated learning materials about Generative AI and Large Language Models (LLMs) such as ChatGPT</a> (2023)</li><li><a href="https://www.andrewthompson.co/2017/05/yoyo-wallet-powers-new-caffe-nero-app.html">Yoyo Wallet Powers The New Caffè Nero App</a> (2017)</li>
<li><a href="https://www.andrewthompson.co/2017/07/building-job-finder-skill-for-my-amazon.html">Building a "Job Finder" skill for my Amazon Alexa Device</a> (2017)</li>
<li><a href="http://www.andrewthompson.co/2012/12/prototyping-with-googles-prediction-api.html">Prototyping with Google's Prediction API, Python, SQLite and some JSON APIs</a> (2012)</li>
<li>
<div class="p1">
<a href="http://www.andrewthompson.co/2011/12/deploying-packages-to-azure-compute.html">Deploying Packages to the Azure Compute Emulator for Automated Testing across Local Environments and a Continuous Integration (CI) Server</a> (2011)</div>
</li>
<li><div class="p1">
<a href="http://www.andrewthompson.co/2011/06/configuring-androids-webview-component.html">Configuring an Android Application's WebView Component</a> (2011)</div>
</li>
<li><div class="p1">
<a href="http://www.andrewthompson.co/2011/05/technical-debt.html">Technical Debt</a> (2011)</div>
</li>
<li><div class="p1">
<a href="http://www.andrewthompson.co/2011/03/achieving-transactions-with-database.html">Achieving Transactions with a Database and File System</a> (2011)</div>
</li>
<li><div class="p1">
<a href="http://www.andrewthompson.co/2011/02/theory-of-software-development-bugs.html">A Theory of Software Development: Bugs</a> (2011)</div>
</li>
<li><div class="p1">
<a href="http://www.andrewthompson.co/2011/02/byte-order-mark-found-using-net.html">Byte Order Mark found using .NET BinaryReader class</a> (2011)</div></li></ul><div><ul style="text-align: left;">
</ul>
<div>
<br /></div>
</div>
<h2 style="text-align: left;">
Product Growth</h2>
<ul style="text-align: left;">
<li><a href="http://www.andrewthompson.co/2014/06/how-one-word-change-increased-product.html">How a one word change increased product demo conversions by 139%</a> (2014)</li>
<li><a href="http://www.andrewthompson.co/2014/09/improving-on-site-search-functionality.html">Improving On-Site Search Functionality For GrowthHackers.com</a> (2014)</li>
<li><a href="http://www.andrewthompson.co/2013/08/3-responsibilities-of-product-management.html">3 Responsibilities of Product Management</a> (2013)</li>
<li><a href="http://www.andrewthompson.co/2013/12/running-ab-tests-on-mobile-apps.html">10 Steps For Running A/B Tests On Mobile Apps</a> (2013)</li>
<li><a href="http://www.andrewthompson.co/2013/11/a-growth-hacking-case-study-on.html">A Growth Hacking Case Study on Starbucks SRCH</a> (2013)</li>
<li><a href="http://www.andrewthompson.co/2013/09/examples-of-mobile-first-development.html">Examples of Mobile First Development</a> (2013)</li>
<li><a href="http://www.andrewthompson.co/2013/07/mixpanel-implementation-of-startup.html">Mixpanel Implementation of Startup Metrics for Pirates: AARRR</a> (2013)</li>
<li><a href="http://www.andrewthompson.co/2013/05/transitioning-from-engineering-to.html">Transitioning from Engineering to Growth Hacking</a> (2013)</li>
<li><a href="http://www.andrewthompson.co/2013/04/chamath-palihapitiya-vp-of-growth.html">Chamath Palihapitiya (VP of Growth @ Facebook) on focusing on the right things</a> (2013)</li>
</ul>
<div>
<br /></div>
<div>
<div>
<h2 style="text-align: left;">
Events</h2>
</div>
<div>
<ul style="text-align: left;">
<li><a href="https://www.andrewthompson.co/2023/06/generative-ai-opportunities-and-risks.html">Generative AI: Opportunities and risks for property transactions</a> (2023)</li><li><a href="https://www.andrewthompson.co/2022/01/building-software-that-thinks-like.html">Building software that 'thinks' like a lawyer</a> (2021)</li><li><a href="https://www.andrewthompson.co/2020/02/interviewing-benedict-evans-at-appear.html">Interviewing Benedict Evans at Appear Here's Global Gathering</a> (2020)</li>
<li><a href="https://www.andrewthompson.co/2019/10/speaking-at-movers-shakers-property.html">Speaking at Movers & Shakers Property Networking Forum</a> (2019)</li>
<li><a href="https://www.andrewthompson.co/2019/04/ldcs-19th-retail-and-leisure-summit.html">Speaking at LDC's 19th Retail and Leisure Summit Event</a> (2019)</li>
<li><a href="https://www.andrewthompson.co/2018/09/speaking-at-facebook-leadership-summit.html">Speaking at Facebook's Leadership Summit & Hired's Employer Brand Health Panel</a> (2018)</li>
<li><a href="http://www.andrewthompson.co/2012/01/presenting-at-vancouver-windows-azure.html">Presenting at the Vancouver Windows Azure Meetup Group hosted by Microsoft</a> (2012)</li>
</ul>
</div>
</div>
</div>
Andrewhttp://www.blogger.com/profile/03252749023463514361noreply@blogger.com0tag:blogger.com,1999:blog-1847717784907730054.post-29824661961608990242022-04-25T20:37:00.002+01:002022-04-27T00:14:39.470+01:00Flexible working at Orbital Witness<h3 style="text-align: left;"><span style="font-size: medium;">What does 'flexible working' actually look like at our company?</span></h3><div><span style="font-size: small;"><br /></span></div><div>This is a <a href="https://tech.orbitalwitness.com/posts/2022-04-25-flexible-working-at-orbital-witness/">repost</a> from <a href="https://tech.orbitalwitness.com/">Orbital Witness's Tech Blog</a> that I recently setup with the engineering and data science teams to both recognise the work colleagues were doing internally as well as promote their work externally to attract potential candidates when hiring. </div><div><br class="Apple-interchange-newline" /><hr /></div><div style="text-align: justify;"><div class="separator" style="clear: both; text-align: left;"><br /></div><div class="separator" style="clear: both; text-align: left;"><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/a/AVvXsEhL0v47RTripXQ1MXJtqQAxWtYIV4qEFvpjyEc4zVXo2VTZOkwUJ6YSCfySgTFswZRhfbGTG_LIKiIkQvTd6QOta7xoeu7vwCklSpiT3oc8y2zukOHsNiEsG_Bu52aVGsIXtS2JF2BBIRQOscUfokH-DWWbU9C38CXOF080m039Mh7aNbPECCtR3bVPSQ" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><span style="color: black;"><img alt="" data-original-height="726" data-original-width="982" height="474" src="https://blogger.googleusercontent.com/img/a/AVvXsEhL0v47RTripXQ1MXJtqQAxWtYIV4qEFvpjyEc4zVXo2VTZOkwUJ6YSCfySgTFswZRhfbGTG_LIKiIkQvTd6QOta7xoeu7vwCklSpiT3oc8y2zukOHsNiEsG_Bu52aVGsIXtS2JF2BBIRQOscUfokH-DWWbU9C38CXOF080m039Mh7aNbPECCtR3bVPSQ=w640-h474" width="640" /></span></a></div><br /><br /></div><h3 style="clear: both; text-align: left;"><span style="font-family: inherit; font-size: large;"><br /></span></h3><h3 style="clear: both; text-align: left;"><span style="font-family: inherit; font-size: large;">Clarifying the ambiguity</span></h3><div class="separator" style="clear: both; text-align: left;"><p style="--tw-border-opacity: 1; --tw-ring-offset-shadow: 0 0 #0000; --tw-ring-shadow: 0 0 #0000; --tw-shadow: 0 0 #0000; border-color: rgba(229, 231, 235, var(--tw-border-opacity)); border-style: solid; border-width: 0px; box-sizing: border-box; line-height: 30px; margin: 0px 0px 1.5rem;"><span style="font-family: inherit;">Remote working, working from home (WFH), hybrid working, flexible working, cross-border working, mandatory days, offsites, in-person, distributed… on and on the list goes 🤔. There’s a plethora of ways companies and employees are defining how they want to work and what’s expected of everyone while at work. Depending on your previous work environment or what you’ve read about how other people are “working flexibly”, it can mean many different things to different people at different times.</span></p><p style="--tw-border-opacity: 1; --tw-ring-offset-shadow: 0 0 #0000; --tw-ring-shadow: 0 0 #0000; --tw-shadow: 0 0 #0000; border-color: rgba(229, 231, 235, var(--tw-border-opacity)); border-style: solid; border-width: 0px; box-sizing: border-box; line-height: 30px; margin: 0px 0px 1.5rem;"><span style="font-family: inherit;">Having a blanket policy that works in all situations for all people is difficult to say the least. Instead of having such a policy that covers all possible scenarios, we instead took the approach to gather feedback from employees and combine that with what we value and want to foster as a company long-term. From this more organic approach, a set of day-to-day flexible working practices have emerged which employees find works well for them both personally and professionally. This blog post aims to highlight many of those flexible working practices; both to codify them for our own internal use but also to share them with potential future employees who want a sneak peek into how we work.</span></p><hr style="--tw-border-opacity: 1; --tw-ring-offset-shadow: 0 0 #0000; --tw-ring-shadow: 0 0 #0000; --tw-shadow: 0 0 #0000; align-self: center; border: none; box-sizing: border-box; height: 2px; margin: 1.5rem 0px; width: 6rem;" /><h3 style="--tw-border-opacity: 1; --tw-ring-offset-shadow: 0 0 #0000; --tw-ring-shadow: 0 0 #0000; --tw-shadow: 0 0 #0000; border-color: rgba(229, 231, 235, var(--tw-border-opacity)); border-style: solid; border-width: 0px; box-sizing: border-box; margin: 0px 0px 0.75rem; text-align: left;"><span style="font-family: inherit; font-size: large;">Asking employees for their feedback</span></h3><p style="--tw-border-opacity: 1; --tw-ring-offset-shadow: 0 0 #0000; --tw-ring-shadow: 0 0 #0000; --tw-shadow: 0 0 #0000; border-color: rgba(229, 231, 235, var(--tw-border-opacity)); border-style: solid; border-width: 0px; box-sizing: border-box; line-height: 30px; margin: 0px 0px 1.5rem;"><span style="font-family: inherit;">In May 2021, while the UK was still under a partial lockdown, we surveyed employees about what they’d like to see in the next 6 months in terms of flexible working. We wanted to use the responses to create a consistent company policy that worked for everyone. We wanted to learn more about what we had been hearing from some employees: that they were missing elements of a working environment when forced to work from home for long periods of time during the pandemic. Here are the survey results off the back of asking everyone in the company this question: <span style="background-color: #cfe2f3;"><code style="--tw-border-opacity: 1; --tw-ring-offset-shadow: 0 0 #0000; --tw-ring-shadow: 0 0 #0000; --tw-shadow: 0 0 #0000; border-color: rgba(229, 231, 235, var(--tw-border-opacity)); border-style: solid; border-width: 0px; box-sizing: border-box; padding: 2px 4px;">Over the first 6 months from lockdown lifting (21 June, 2021), how many days a week, on average, would you ideally want to be in the office?</code>:</span></span></p><p style="--tw-border-opacity: 1; --tw-ring-offset-shadow: 0 0 #0000; --tw-ring-shadow: 0 0 #0000; --tw-shadow: 0 0 #0000; border-color: rgba(229, 231, 235, var(--tw-border-opacity)); border-style: solid; border-width: 0px; box-sizing: border-box; line-height: 30px; margin: 0px 0px 1.5rem;"><span style="font-family: inherit;"></span></p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/a/AVvXsEjWPrceV2YiRWCqMOnbnCvw5lDQEGqSKYtSi0aylm3nqYTgsD-j8qxw3VkYhMQH0-iXASA0xm9t7iW5m0-2W1FU__zUhgI6_sIi6bZguMtLBHEMMtXTBeiidfyVWQMFN-yqpawKAmb7NvMKjJFhD7YDtC6lbRwGFgaQexvFAmlVs1PlanLytUdBrq5vMQ" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><span style="color: black;"><img alt="" data-original-height="1616" data-original-width="2880" height="360" src="https://blogger.googleusercontent.com/img/a/AVvXsEjWPrceV2YiRWCqMOnbnCvw5lDQEGqSKYtSi0aylm3nqYTgsD-j8qxw3VkYhMQH0-iXASA0xm9t7iW5m0-2W1FU__zUhgI6_sIi6bZguMtLBHEMMtXTBeiidfyVWQMFN-yqpawKAmb7NvMKjJFhD7YDtC6lbRwGFgaQexvFAmlVs1PlanLytUdBrq5vMQ=w640-h360" width="640" /></span></a><span style="font-family: inherit;"></span></div><span style="font-family: inherit;"><br /><br /></span><p></p><p style="--tw-border-opacity: 1; --tw-ring-offset-shadow: 0 0 #0000; --tw-ring-shadow: 0 0 #0000; --tw-shadow: 0 0 #0000; border-color: rgba(229, 231, 235, var(--tw-border-opacity)); border-style: solid; border-width: 0px; box-sizing: border-box; font-family: Poppins, ui-sans-serif, system-ui, -apple-system, "system-ui", "Segoe UI", Roboto, "Helvetica Neue", Arial, "Noto Sans", sans-serif, "Apple Color Emoji", "Segoe UI Emoji", "Segoe UI Symbol", "Noto Color Emoji"; line-height: 30px; margin: 0px 0px 1.5rem;"><br /></p><p style="--tw-border-opacity: 1; --tw-ring-offset-shadow: 0 0 #0000; --tw-ring-shadow: 0 0 #0000; --tw-shadow: 0 0 #0000; border-color: rgba(229, 231, 235, var(--tw-border-opacity)); border-style: solid; border-width: 0px; box-sizing: border-box; font-family: Poppins, ui-sans-serif, system-ui, -apple-system, "system-ui", "Segoe UI", Roboto, "Helvetica Neue", Arial, "Noto Sans", sans-serif, "Apple Color Emoji", "Segoe UI Emoji", "Segoe UI Symbol", "Noto Color Emoji"; line-height: 30px; margin: 0px 0px 1.5rem;"><br /></p><p style="--tw-border-opacity: 1; --tw-ring-offset-shadow: 0 0 #0000; --tw-ring-shadow: 0 0 #0000; --tw-shadow: 0 0 #0000; border-color: rgba(229, 231, 235, var(--tw-border-opacity)); border-style: solid; border-width: 0px; box-sizing: border-box; font-family: Poppins, ui-sans-serif, system-ui, -apple-system, "system-ui", "Segoe UI", Roboto, "Helvetica Neue", Arial, "Noto Sans", sans-serif, "Apple Color Emoji", "Segoe UI Emoji", "Segoe UI Symbol", "Noto Color Emoji"; line-height: 30px; margin: 0px 0px 1.5rem;"><br /></p><p style="--tw-border-opacity: 1; --tw-ring-offset-shadow: 0 0 #0000; --tw-ring-shadow: 0 0 #0000; --tw-shadow: 0 0 #0000; border-color: rgba(229, 231, 235, var(--tw-border-opacity)); border-style: solid; border-width: 0px; box-sizing: border-box; font-family: Poppins, ui-sans-serif, system-ui, -apple-system, "system-ui", "Segoe UI", Roboto, "Helvetica Neue", Arial, "Noto Sans", sans-serif, "Apple Color Emoji", "Segoe UI Emoji", "Segoe UI Symbol", "Noto Color Emoji"; line-height: 30px; margin: 0px 0px 1.5rem;"><br /></p><p style="--tw-border-opacity: 1; --tw-ring-offset-shadow: 0 0 #0000; --tw-ring-shadow: 0 0 #0000; --tw-shadow: 0 0 #0000; border-color: rgba(229, 231, 235, var(--tw-border-opacity)); border-style: solid; border-width: 0px; box-sizing: border-box; font-family: Poppins, ui-sans-serif, system-ui, -apple-system, "system-ui", "Segoe UI", Roboto, "Helvetica Neue", Arial, "Noto Sans", sans-serif, "Apple Color Emoji", "Segoe UI Emoji", "Segoe UI Symbol", "Noto Color Emoji"; line-height: 30px; margin: 0px 0px 1.5rem;"><br /></p><p style="--tw-border-opacity: 1; --tw-ring-offset-shadow: 0 0 #0000; --tw-ring-shadow: 0 0 #0000; --tw-shadow: 0 0 #0000; border-color: rgba(229, 231, 235, var(--tw-border-opacity)); border-style: solid; border-width: 0px; box-sizing: border-box; font-family: Poppins, ui-sans-serif, system-ui, -apple-system, "system-ui", "Segoe UI", Roboto, "Helvetica Neue", Arial, "Noto Sans", sans-serif, "Apple Color Emoji", "Segoe UI Emoji", "Segoe UI Symbol", "Noto Color Emoji"; line-height: 30px; margin: 0px 0px 1.5rem;"><br /></p><p style="--tw-border-opacity: 1; --tw-ring-offset-shadow: 0 0 #0000; --tw-ring-shadow: 0 0 #0000; --tw-shadow: 0 0 #0000; border-color: rgba(229, 231, 235, var(--tw-border-opacity)); border-style: solid; border-width: 0px; box-sizing: border-box; line-height: 30px; margin: 0px 0px 1.5rem;"><span style="font-family: inherit;">Along with collecting the quantitative data above 📊, we asked a few follow-up questions about what people value most about working from home and working in the office to help us understand how we could foster the right working environment for both settings. We created the word clouds seen below from everyone’s responses which showed us the different aspects of each environment that are valued by our current employees (the bigger the word, the more often it was mentioned by everyone):</span></p><table style="--tw-border-opacity: 1; --tw-ring-offset-shadow: 0 0 #0000; --tw-ring-shadow: 0 0 #0000; --tw-shadow: 0 0 #0000; border-collapse: collapse; border-color: inherit; border-style: solid; border-width: 0px; font-family: Poppins, ui-sans-serif, system-ui, -apple-system, "system-ui", "Segoe UI", Roboto, "Helvetica Neue", Arial, "Noto Sans", sans-serif, "Apple Color Emoji", "Segoe UI Emoji", "Segoe UI Symbol", "Noto Color Emoji"; text-indent: 0px;"><tbody style="--tw-border-opacity: 1; --tw-ring-offset-shadow: 0 0 #0000; --tw-ring-shadow: 0 0 #0000; --tw-shadow: 0 0 #0000; border-color: rgba(229, 231, 235, var(--tw-border-opacity)); border-style: solid; border-width: 0px; box-sizing: border-box;"><tr style="--tw-border-opacity: 1; --tw-ring-offset-shadow: 0 0 #0000; --tw-ring-shadow: 0 0 #0000; --tw-shadow: 0 0 #0000; border-color: rgba(229, 231, 235, var(--tw-border-opacity)); border-style: solid; border-width: 0px; box-sizing: border-box;"><th style="--tw-border-opacity: 1; --tw-ring-offset-shadow: 0 0 #0000; --tw-ring-shadow: 0 0 #0000; --tw-shadow: 0 0 #0000; border-color: rgba(229, 231, 235, var(--tw-border-opacity)); border-style: solid; border-width: 0px; box-sizing: border-box;"><span style="font-family: inherit;">Reasons for working from home</span></th><th style="--tw-border-opacity: 1; --tw-ring-offset-shadow: 0 0 #0000; --tw-ring-shadow: 0 0 #0000; --tw-shadow: 0 0 #0000; border-color: rgba(229, 231, 235, var(--tw-border-opacity)); border-style: solid; border-width: 0px; box-sizing: border-box;"><span style="font-family: inherit;">Reasons for working in the office</span></th></tr><tr style="--tw-border-opacity: 1; --tw-ring-offset-shadow: 0 0 #0000; --tw-ring-shadow: 0 0 #0000; --tw-shadow: 0 0 #0000; border-color: rgba(229, 231, 235, var(--tw-border-opacity)); border-style: solid; border-width: 0px; box-sizing: border-box;"><td style="--tw-border-opacity: 1; --tw-ring-offset-shadow: 0 0 #0000; --tw-ring-shadow: 0 0 #0000; --tw-shadow: 0 0 #0000; border-color: rgba(229, 231, 235, var(--tw-border-opacity)); border-style: solid; border-width: 0px; box-sizing: border-box;"><figure style="--tw-border-opacity: 1; --tw-ring-offset-shadow: 0 0 #0000; --tw-ring-shadow: 0 0 #0000; --tw-shadow: 0 0 #0000; align-items: center; border-color: rgba(229, 231, 235, var(--tw-border-opacity)); border-style: solid; border-width: 0px; box-sizing: border-box; display: flex; flex-direction: column; margin: 0px;"><img alt="Reasons for working from home" src="https://tech.orbitalwitness.com/images/flexible-working-at-orbital-witness/reasons-from-home.png" style="--tw-border-opacity: 1; --tw-ring-offset-shadow: 0 0 #0000; --tw-ring-shadow: 0 0 #0000; --tw-shadow: 0 0 #0000; border-color: rgba(229, 231, 235, var(--tw-border-opacity)); border-style: solid; border-width: 0px; box-sizing: border-box; display: block; height: auto; max-width: 100%; vertical-align: middle;" /></figure></td><td style="--tw-border-opacity: 1; --tw-ring-offset-shadow: 0 0 #0000; --tw-ring-shadow: 0 0 #0000; --tw-shadow: 0 0 #0000; border-color: rgba(229, 231, 235, var(--tw-border-opacity)); border-style: solid; border-width: 0px; box-sizing: border-box;"><figure style="--tw-border-opacity: 1; --tw-ring-offset-shadow: 0 0 #0000; --tw-ring-shadow: 0 0 #0000; --tw-shadow: 0 0 #0000; align-items: center; border-color: rgba(229, 231, 235, var(--tw-border-opacity)); border-style: solid; border-width: 0px; box-sizing: border-box; display: flex; flex-direction: column; margin: 0px;"><img alt="Reasons for working in the office" src="https://tech.orbitalwitness.com/images/flexible-working-at-orbital-witness/reasons-from-office.png" style="--tw-border-opacity: 1; --tw-ring-offset-shadow: 0 0 #0000; --tw-ring-shadow: 0 0 #0000; --tw-shadow: 0 0 #0000; border-color: rgba(229, 231, 235, var(--tw-border-opacity)); border-style: solid; border-width: 0px; box-sizing: border-box; display: block; height: auto; max-width: 100%; vertical-align: middle;" /></figure></td></tr></tbody></table><hr style="--tw-border-opacity: 1; --tw-ring-offset-shadow: 0 0 #0000; --tw-ring-shadow: 0 0 #0000; --tw-shadow: 0 0 #0000; align-self: center; border: none; box-sizing: border-box; font-family: Poppins, ui-sans-serif, system-ui, -apple-system, "system-ui", "Segoe UI", Roboto, "Helvetica Neue", Arial, "Noto Sans", sans-serif, "Apple Color Emoji", "Segoe UI Emoji", "Segoe UI Symbol", "Noto Color Emoji"; height: 2px; margin: 1.5rem 0px; width: 6rem;" /><h3 style="--tw-border-opacity: 1; --tw-ring-offset-shadow: 0 0 #0000; --tw-ring-shadow: 0 0 #0000; --tw-shadow: 0 0 #0000; border-color: rgba(229, 231, 235, var(--tw-border-opacity)); border-style: solid; border-width: 0px; box-sizing: border-box; margin: 0px 0px 0.75rem; text-align: left;"><span style="font-family: inherit; font-size: large;">What we value and what we’re trying to foster</span></h3><p style="--tw-border-opacity: 1; --tw-ring-offset-shadow: 0 0 #0000; --tw-ring-shadow: 0 0 #0000; --tw-shadow: 0 0 #0000; border-color: rgba(229, 231, 235, var(--tw-border-opacity)); border-style: solid; border-width: 0px; box-sizing: border-box; line-height: 30px; margin: 0px 0px 1.5rem;"><span style="font-family: inherit;">At the heart of <a href="https://tech.orbitalwitness.com/posts/2021-11-02-what-its-like-to-work-at-ow" style="--tw-border-opacity: 1; --tw-ring-offset-shadow: 0 0 #0000; --tw-ring-shadow: 0 0 #0000; --tw-shadow: 0 0 #0000; border-color: rgba(229, 231, 235, var(--tw-border-opacity)); border-style: solid; border-width: 0px; box-sizing: border-box;">our culture and values</a> - most notably, Power to Our People - is the <code style="--tw-border-opacity: 1; --tw-ring-offset-shadow: 0 0 #0000; --tw-ring-shadow: 0 0 #0000; --tw-shadow: 0 0 #0000; background-color: #cfe2f3; border-color: rgba(229, 231, 235, var(--tw-border-opacity)); border-style: solid; border-width: 0px; box-sizing: border-box; padding: 2px 4px;">trust</code> we place in our employees. We enjoy giving everyone both the <code style="--tw-border-opacity: 1; --tw-ring-offset-shadow: 0 0 #0000; --tw-ring-shadow: 0 0 #0000; --tw-shadow: 0 0 #0000; border-color: rgba(229, 231, 235, var(--tw-border-opacity)); border-style: solid; border-width: 0px; box-sizing: border-box; padding: 2px 4px;">autonomy</code> to decide which work environment is the most conducive for them to do their best work, and the <code style="--tw-border-opacity: 1; --tw-ring-offset-shadow: 0 0 #0000; --tw-ring-shadow: 0 0 #0000; --tw-shadow: 0 0 #0000; background-color: #cfe2f3; border-color: rgba(229, 231, 235, var(--tw-border-opacity)); border-style: solid; border-width: 0px; box-sizing: border-box; padding: 2px 4px;">flexibility</code> 🧘♀️🙆♂️ to put this into practice today, tomorrow or in the future.</span></p><p style="--tw-border-opacity: 1; --tw-ring-offset-shadow: 0 0 #0000; --tw-ring-shadow: 0 0 #0000; --tw-shadow: 0 0 #0000; border-color: rgba(229, 231, 235, var(--tw-border-opacity)); border-style: solid; border-width: 0px; box-sizing: border-box; line-height: 30px; margin: 0px 0px 1.5rem;"><span style="font-family: inherit;">At the same time, we also want to foster a thriving, collaborative, and pleasant working environment for those employees that have a specific need or desire for home or office working. Each team and each person works differently depending on their needs. Sometimes, the number of days worked from home or in the office may be a consistent trend, but it may also become sporadic and less predictable due to personal circumstances or the nature of the specific project they’re working on. The guiding principle we follow when allowing for flexibility is <code style="--tw-border-opacity: 1; --tw-ring-offset-shadow: 0 0 #0000; --tw-ring-shadow: 0 0 #0000; --tw-shadow: 0 0 #0000; background-color: #cfe2f3; border-color: rgba(229, 231, 235, var(--tw-border-opacity)); border-style: solid; border-width: 0px; box-sizing: border-box; padding: 2px 4px;">Customers => Colleagues => Self</code>. This means that if we are ever in doubt about what to prioritise then whatever is best for the customer comes first, followed by what’s best for our colleagues as a team, followed by what’s best according to one’s personal preferences. This principle irons out most issues and allows everyone to work as flexibly as they need according to their task(s) while aligning everyone to what’s most important for the business to continue thriving.</span></p><p style="--tw-border-opacity: 1; --tw-ring-offset-shadow: 0 0 #0000; --tw-ring-shadow: 0 0 #0000; --tw-shadow: 0 0 #0000; border-color: rgba(229, 231, 235, var(--tw-border-opacity)); border-style: solid; border-width: 0px; box-sizing: border-box; line-height: 30px; margin: 0px 0px 1.5rem;"><span style="font-family: inherit;">The desire for flexibility was not only evident in much of the survey data shown previously, but also in one-on-one meetings with colleagues and in interviews where we listened 👂 to what potential candidates wanted in their future work setup and/or thought were missing from their current work environment. Pretty much everyone loves the comfort, convenience and solitude that home working provides but they also miss the structure/routine, contact with colleagues, and dynamism that multiple colleagues in an office can provide. At Orbital Witness we wanted to have our cake and eat it too 🍰 and therefore we actively foster the ability for any employee to choose either working option - on any given day. Sometimes people choose both options on the same day such as working from home in the morning and from the office in the afternoon!</span></p><p style="--tw-border-opacity: 1; --tw-ring-offset-shadow: 0 0 #0000; --tw-ring-shadow: 0 0 #0000; --tw-shadow: 0 0 #0000; border-color: rgba(229, 231, 235, var(--tw-border-opacity)); border-style: solid; border-width: 0px; box-sizing: border-box; line-height: 30px; margin: 0px 0px 1.5rem;"><span style="font-family: inherit;">Everyone is different and therefore requires a different setup to be productive and happy at their job. A 25 year old living in a cramped London flatshare 🏢 may need the respite and space provided by a quiet office environment with meeting rooms. However, a parent of two little ones 👶🧒 needing to conduct the school drop-off may find working from home or simply some more flexible time in the mornings really helpful to start the day off on the right foot. Then there’s also those times where someone just needs to find the time and space to get into the zone and spend hours of uninterrupted time solving a hard technical problem.</span></p><p style="--tw-border-opacity: 1; --tw-ring-offset-shadow: 0 0 #0000; --tw-ring-shadow: 0 0 #0000; --tw-shadow: 0 0 #0000; border-color: rgba(229, 231, 235, var(--tw-border-opacity)); border-style: solid; border-width: 0px; box-sizing: border-box; line-height: 30px; margin: 0px 0px 1.5rem;"><span style="font-family: inherit;">An added little bonus of working at Orbital Witness is that we currently have an office in a building operated by The Office Group (TOG) which is similar to a WeWork concept. Because of this affiliation with TOG, every Orbital Witness employee not only has their own home and our usual permanent office environment in Clerkenwell to work from but also <a href="https://www.theofficegroup.com/office-space/uk/london" style="--tw-border-opacity: 1; --tw-ring-offset-shadow: 0 0 #0000; --tw-ring-shadow: 0 0 #0000; --tw-shadow: 0 0 #0000; border-color: rgba(229, 231, 235, var(--tw-border-opacity)); border-style: solid; border-width: 0px; box-sizing: border-box;">40+ TOG buildings throughout London</a> (and even more offices in other cities in the UK and Europe) which they have access to with their TOG key card.</span></p><h3 style="--tw-border-opacity: 1; --tw-ring-offset-shadow: 0 0 #0000; --tw-ring-shadow: 0 0 #0000; --tw-shadow: 0 0 #0000; border-color: rgba(229, 231, 235, var(--tw-border-opacity)); border-style: solid; border-width: 0px; box-sizing: border-box; margin: 0px 0px 0.75rem; text-align: left;"><span style="font-family: inherit; font-size: large;">What’s actually happening day-to-day</span></h3><h4 style="--tw-border-opacity: 1; --tw-ring-offset-shadow: 0 0 #0000; --tw-ring-shadow: 0 0 #0000; --tw-shadow: 0 0 #0000; border-color: rgba(229, 231, 235, var(--tw-border-opacity)); border-style: solid; border-width: 0px; box-sizing: border-box; margin: 0px 0px 0.75rem; text-align: left;"><span style="font-family: inherit; font-size: medium;">Flexible days of the week</span></h4><p style="--tw-border-opacity: 1; --tw-ring-offset-shadow: 0 0 #0000; --tw-ring-shadow: 0 0 #0000; --tw-shadow: 0 0 #0000; border-color: rgba(229, 231, 235, var(--tw-border-opacity)); border-style: solid; border-width: 0px; box-sizing: border-box; line-height: 30px; margin: 0px 0px 1.5rem;"><span style="font-family: inherit;">There isn’t a set number of days we ask everyone to be in the office and instead allow for more of an organic approach. So what’s actually going on in a given week? Here is data taken from our <a href="https://orbitalwitness.slack.com/apps/A018RKTRY8P-officely" style="--tw-border-opacity: 1; --tw-ring-offset-shadow: 0 0 #0000; --tw-ring-shadow: 0 0 #0000; --tw-shadow: 0 0 #0000; border-color: rgba(229, 231, 235, var(--tw-border-opacity)); border-style: solid; border-width: 0px; box-sizing: border-box;">Officely Slack App</a> which collects self-reported data from employees about when they’re coming into the office. Colleagues use Officely each week to inform everyone else about when they might be coming into the office on a specific day. Officely helps us all to more effectively coordinate meetings and have impromptu meetups between colleagues in the same team or across teams. Officely has been really helpful to make sure more of our one-on-one meetings happen in person over a coffee ☕️, while our communal lunches spill over many tables and after-work pub get-togethers with colleagues are a staple. Clearly, Wednesdays and Thursdays are good days to coordinate those things, with Friday being a little less busy:</span></p><p style="--tw-border-opacity: 1; --tw-ring-offset-shadow: 0 0 #0000; --tw-ring-shadow: 0 0 #0000; --tw-shadow: 0 0 #0000; border-color: rgba(229, 231, 235, var(--tw-border-opacity)); border-style: solid; border-width: 0px; box-sizing: border-box; line-height: 30px; margin: 0px 0px 1.5rem;"></p><div class="separator" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em; text-align: center;"><a href="https://blogger.googleusercontent.com/img/a/AVvXsEhIsaxj86nsARzSfVY9EKt1lFTRc61ce5c2rgOSo-EywMkDNAFbu_HHSUKWM_DTOFzzulcl5I5c6QQMQU-MKm_QUo3VYqiHoZoskGOqO4ju8joLzpiB9bw4pgDx5ZYkXrrjEPuJJePMGXLNN_qHmC1DLqBGYSX7TaZI4HCPW_GP70-3ie6GKlwRXigB8Q" style="margin-left: 1em; margin-right: 1em;"><span style="color: black;"><img alt="" data-original-height="1618" data-original-width="2880" height="360" src="https://blogger.googleusercontent.com/img/a/AVvXsEhIsaxj86nsARzSfVY9EKt1lFTRc61ce5c2rgOSo-EywMkDNAFbu_HHSUKWM_DTOFzzulcl5I5c6QQMQU-MKm_QUo3VYqiHoZoskGOqO4ju8joLzpiB9bw4pgDx5ZYkXrrjEPuJJePMGXLNN_qHmC1DLqBGYSX7TaZI4HCPW_GP70-3ie6GKlwRXigB8Q=w640-h360" width="640" /></span></a></div><span style="font-family: inherit;"><br /><br /></span><p></p><figure style="--tw-border-opacity: 1; --tw-ring-offset-shadow: 0 0 #0000; --tw-ring-shadow: 0 0 #0000; --tw-shadow: 0 0 #0000; align-items: center; border-color: rgba(229, 231, 235, var(--tw-border-opacity)); border-style: solid; border-width: 0px; box-sizing: border-box; display: flex; flex-direction: column; font-family: Poppins, ui-sans-serif, system-ui, -apple-system, "system-ui", "Segoe UI", Roboto, "Helvetica Neue", Arial, "Noto Sans", sans-serif, "Apple Color Emoji", "Segoe UI Emoji", "Segoe UI Symbol", "Noto Color Emoji"; margin: 0px;"></figure><h3 id="flexible-location" style="--tw-border-opacity: 1; --tw-ring-offset-shadow: 0 0 #0000; --tw-ring-shadow: 0 0 #0000; --tw-shadow: 0 0 #0000; border-color: rgba(229, 231, 235, var(--tw-border-opacity)); border-style: solid; border-width: 0px; box-sizing: border-box; font-family: Poppins, ui-sans-serif, system-ui, -apple-system, "system-ui", "Segoe UI", Roboto, "Helvetica Neue", Arial, "Noto Sans", sans-serif, "Apple Color Emoji", "Segoe UI Emoji", "Segoe UI Symbol", "Noto Color Emoji"; font-weight: 500; margin: 0px 0px 0.75rem;"><span style="font-size: small;"><br /></span></h3><h3 id="flexible-location" style="--tw-border-opacity: 1; --tw-ring-offset-shadow: 0 0 #0000; --tw-ring-shadow: 0 0 #0000; --tw-shadow: 0 0 #0000; border-color: rgba(229, 231, 235, var(--tw-border-opacity)); border-style: solid; border-width: 0px; box-sizing: border-box; margin: 0px 0px 0.75rem;"><span style="font-family: inherit; font-size: medium;">Flexible location</span></h3><p style="--tw-border-opacity: 1; --tw-ring-offset-shadow: 0 0 #0000; --tw-ring-shadow: 0 0 #0000; --tw-shadow: 0 0 #0000; border-color: rgba(229, 231, 235, var(--tw-border-opacity)); border-style: solid; border-width: 0px; box-sizing: border-box; line-height: 30px; margin: 0px 0px 1.5rem;"><span style="font-family: inherit;">The vast majority of employees at Orbital Witness are based in and around London with a few people as far flung as the seaside 🌊 in Kent and the countryside 🏕 near Bristol and Cardiff 🏴. As such, most employees come into the office on a weekly/fortnightly basis for a few or more days. Those who are more far flung come into the office monthly/bi-monthly for a few days at a time, often for our company all-hands once a month. We even have some colleagues who come into the office most of the week because they like the routine and structure of it. To summarise, there’s a complete mix of what people currently do and hence this is why we value continuing to have that flexibility. Our feedback and surveys show that it’s working well both in terms of productivity and employee happiness; therefore, we’d like that to continue 😊.</span></p><p style="--tw-border-opacity: 1; --tw-ring-offset-shadow: 0 0 #0000; --tw-ring-shadow: 0 0 #0000; --tw-shadow: 0 0 #0000; border-color: rgba(229, 231, 235, var(--tw-border-opacity)); border-style: solid; border-width: 0px; box-sizing: border-box; line-height: 30px; margin: 0px 0px 1.5rem;"><span style="font-family: inherit;">Given that we’re trying to foster both flexibility and an in-office culture, we’ve deliberately decided not to seek out fully remote colleagues who are not able to come into the office at all. We’re not dogmatic about this and will make the odd exception to the rule, but in the long-term we want to have a company with a healthy segment of employees who work and socialise in-person on a regular basis, especially while our teams grow and our culture evolves.</span></p><h3 id="flexible-working-hours" style="--tw-border-opacity: 1; --tw-ring-offset-shadow: 0 0 #0000; --tw-ring-shadow: 0 0 #0000; --tw-shadow: 0 0 #0000; border-color: rgba(229, 231, 235, var(--tw-border-opacity)); border-style: solid; border-width: 0px; box-sizing: border-box; margin: 0px 0px 0.75rem;"><span style="font-family: inherit; font-size: medium;">Flexible working hours</span></h3><p style="--tw-border-opacity: 1; --tw-ring-offset-shadow: 0 0 #0000; --tw-ring-shadow: 0 0 #0000; --tw-shadow: 0 0 #0000; border-color: rgba(229, 231, 235, var(--tw-border-opacity)); border-style: solid; border-width: 0px; box-sizing: border-box; line-height: 30px; margin: 0px 0px 1.5rem;"><span style="font-family: inherit;">Our core working hours are <code style="--tw-border-opacity: 1; --tw-ring-offset-shadow: 0 0 #0000; --tw-ring-shadow: 0 0 #0000; --tw-shadow: 0 0 #0000; background-color: #cfe2f3; border-color: rgba(229, 231, 235, var(--tw-border-opacity)); border-style: solid; border-width: 0px; box-sizing: border-box; padding: 2px 4px;">9:00am to 5:30pm</code> but we’re not overly pernickety about following this and we certainly don’t monitor it. Many of the teams have stand-ups that begin at 9:30am and so long as people communicate effectively about when they might not be available during core hours, everyone is free to hop off to a doctor’s appointment or gym session throughout the day. Again, the reason for this is to give everyone flexibility; we fully trust our colleagues to balance both their personal and professional lives and everyone does just that. The reason we’re comfortable with this approach is that, from a management perspective, we follow what Andy Grove famously wrote about in his book, <a href="https://www.amazon.co.uk/High-Output-Management-Andrew-Grove/dp/0679762884" style="--tw-border-opacity: 1; --tw-ring-offset-shadow: 0 0 #0000; --tw-ring-shadow: 0 0 #0000; --tw-shadow: 0 0 #0000; border-color: rgba(229, 231, 235, var(--tw-border-opacity)); border-style: solid; border-width: 0px; box-sizing: border-box;">High Output Management</a>, where he said <code style="--tw-border-opacity: 1; --tw-ring-offset-shadow: 0 0 #0000; --tw-ring-shadow: 0 0 #0000; --tw-shadow: 0 0 #0000; background-color: #cfe2f3; border-color: rgba(229, 231, 235, var(--tw-border-opacity)); border-style: solid; border-width: 0px; box-sizing: border-box; padding: 2px 4px;">measure output not activity</code>. At Orbital Witness, we are also far more concerned with what output is being generated by a team/individual rather than tracking the activity or amount of time that produced it. As a result, we care less about the hours worked and more about the value we’re continuously shipping to our customers.</span></p><h4 style="--tw-border-opacity: 1; --tw-ring-offset-shadow: 0 0 #0000; --tw-ring-shadow: 0 0 #0000; --tw-shadow: 0 0 #0000; border-color: rgba(229, 231, 235, var(--tw-border-opacity)); border-style: solid; border-width: 0px; box-sizing: border-box; margin: 0px 0px 0.75rem; text-align: left;"><span style="font-family: inherit; font-size: medium;">Company get-togethers</span></h4><p style="--tw-border-opacity: 1; --tw-ring-offset-shadow: 0 0 #0000; --tw-ring-shadow: 0 0 #0000; --tw-shadow: 0 0 #0000; border-color: rgba(229, 231, 235, var(--tw-border-opacity)); border-style: solid; border-width: 0px; box-sizing: border-box; line-height: 30px; margin: 0px 0px 1.5rem;"><span style="font-family: inherit;">Once a month, we have our company <code style="--tw-border-opacity: 1; --tw-ring-offset-shadow: 0 0 #0000; --tw-ring-shadow: 0 0 #0000; --tw-shadow: 0 0 #0000; background-color: #cfe2f3; border-color: rgba(229, 231, 235, var(--tw-border-opacity)); border-style: solid; border-width: 0px; box-sizing: border-box; padding: 2px 4px;">All Hands</code> meeting where we go over key performance metrics of the company, discuss one important topic that matters most and often have a fireside chat with a customer, investor, or industry leader. We’re always set up virtually for these meetings so that those who are working from home can participate. However, for the day of the All Hands, we encourage everyone across the company to be in the office where we have lunch together, schedule collaborative meetings within teams and have the company-wide All Hands meeting. Afterwards, we often have a planned social in the evening which has ranged from canoeing 🛶 Hackney Wick canal (and sometimes falling in) to competing in epic two-on-two ping pong 🏓 battles at Bounce to baking and decorating a Great British Bake Off style cake 🍰 in teams of two (and then devouring the cake before we’ve even got home for the night).</span></p><h3 style="--tw-border-opacity: 1; --tw-ring-offset-shadow: 0 0 #0000; --tw-ring-shadow: 0 0 #0000; --tw-shadow: 0 0 #0000; border-color: rgba(229, 231, 235, var(--tw-border-opacity)); border-style: solid; border-width: 0px; box-sizing: border-box; margin: 0px 0px 0.75rem; text-align: left;"><span style="font-family: inherit; font-size: large;">Other small niceties</span></h3><h3 id="dogs-in-the-office" style="--tw-border-opacity: 1; --tw-ring-offset-shadow: 0 0 #0000; --tw-ring-shadow: 0 0 #0000; --tw-shadow: 0 0 #0000; border-color: rgba(229, 231, 235, var(--tw-border-opacity)); border-style: solid; border-width: 0px; box-sizing: border-box; margin: 0px 0px 0.75rem;"><span style="font-family: inherit; font-size: medium;">Dogs in the office</span></h3><p style="--tw-border-opacity: 1; --tw-ring-offset-shadow: 0 0 #0000; --tw-ring-shadow: 0 0 #0000; --tw-shadow: 0 0 #0000; border-color: rgba(229, 231, 235, var(--tw-border-opacity)); border-style: solid; border-width: 0px; box-sizing: border-box; line-height: 30px; margin: 0px 0px 1.5rem;"><span style="font-family: inherit;">Many colleagues decided to get a new family member during the pandemic 🐕 🐩 🦮 and we applauded them for it. We know how important dogs can be for many people and how much love and attention they need during the day. Because of this, people have the option to bring their dog into the office on select days so long as their furry friend is well trained and doesn’t disturb others who are hard at work. We created a little system so dog owners and some colleagues (who for various reasons may not want to be around dogs in the office) can easily coordinate which day(s) dogs may be in the office and everyone can work around this with little hassle.</span></p><h3 id="video-first-meetings" style="--tw-border-opacity: 1; --tw-ring-offset-shadow: 0 0 #0000; --tw-ring-shadow: 0 0 #0000; --tw-shadow: 0 0 #0000; border-color: rgba(229, 231, 235, var(--tw-border-opacity)); border-style: solid; border-width: 0px; box-sizing: border-box; margin: 0px 0px 0.75rem;"><span style="font-family: inherit; font-size: medium;">Video-first meetings</span></h3><p style="--tw-border-opacity: 1; --tw-ring-offset-shadow: 0 0 #0000; --tw-ring-shadow: 0 0 #0000; --tw-shadow: 0 0 #0000; border-color: rgba(229, 231, 235, var(--tw-border-opacity)); border-style: solid; border-width: 0px; box-sizing: border-box; line-height: 30px; margin: 0px 0px 1.5rem;"><span style="font-family: inherit;">The entire company has weekly stand-ups (Monday mornings) and stand-downs (Friday afternoons) where we briefly talk about the week ahead and/or what was accomplished, the tracking of our OKRs, and any big events/happenings that week. It’s a good touchstone for everyone to see the entire company each week and get a sense for what’s going on. Each team usually has a separate stand-up each day (focused on the specific details of their work) as well as other ad hoc product, design, engineering and/or one-on-one meetings. All of our meetings are set up so that colleagues can either video conference in 🧑💻 or be in the office around a single screen 📺 with good audio quality. As such, most people keep their cameras on as it’s nice to see everyone’s face during these meetings (unless of course people happen to be eating 🍔 or are having a bad hair day 👱♀️).</span></p><h4 style="--tw-border-opacity: 1; --tw-ring-offset-shadow: 0 0 #0000; --tw-ring-shadow: 0 0 #0000; --tw-shadow: 0 0 #0000; border-color: rgba(229, 231, 235, var(--tw-border-opacity)); border-style: solid; border-width: 0px; box-sizing: border-box; margin: 0px 0px 0.75rem; text-align: left;"><span style="font-family: inherit; font-size: medium;">Working remotely for a period of time</span></h4><p style="--tw-border-opacity: 1; --tw-ring-offset-shadow: 0 0 #0000; --tw-ring-shadow: 0 0 #0000; --tw-shadow: 0 0 #0000; border-color: rgba(229, 231, 235, var(--tw-border-opacity)); border-style: solid; border-width: 0px; box-sizing: border-box; line-height: 30px; margin: 0px 0px 1.5rem;"><span style="font-family: inherit;">We have a very international team and that means people’s families and loved ones are spread far and wide across the globe. Coming out of the pandemic, when travel was allowed again, it was vital that everyone was given the opportunity to see their family if they wanted or needed to. We had several colleagues fly off to their home countries such as Croatia 🇭🇷, Brazil 🇧🇷, Malaysia 🇲🇾 and Hungary 🇭🇺 for extended periods of time where they both took holidays and worked remotely for a number of weeks. This allowed them to spend much needed time with family and friends who they hadn’t seen for some time, while also enabling them to continue working on their projects without using up all their holiday allowance in one single shot. We have to be a bit careful to make sure everyone abides by UK tax laws (which dictate how much time someone can work outside the UK while being on a UK payroll), but so long as that’s satisfied, employees wanting to work abroad for up to 8 weeks a year is something that we support. We just ask that employees continue to overlap with the core working hours of their team. If you’re from France that’s a pretty easy trade-off but it can be more difficult the farther afield you travel (depending of course on your tolerance for changing when your work day starts and ends). Having said that, even one of our tech leads, who recently went to visit family and friends in Malaysia 🇲🇾, and one of our senior engineers, who recently went to Brazil 🇧🇷, didn’t mind too much about starting work in the late afternoon or early morning (respectively) to accommodate this.</span></p><h3 style="--tw-border-opacity: 1; --tw-ring-offset-shadow: 0 0 #0000; --tw-ring-shadow: 0 0 #0000; --tw-shadow: 0 0 #0000; border-color: rgba(229, 231, 235, var(--tw-border-opacity)); border-style: solid; border-width: 0px; box-sizing: border-box; margin: 0px 0px 0.75rem; text-align: left;"><span style="font-family: inherit; font-size: large;">The future is adaptable</span></h3><p style="--tw-border-opacity: 1; --tw-ring-offset-shadow: 0 0 #0000; --tw-ring-shadow: 0 0 #0000; --tw-shadow: 0 0 #0000; border-color: rgba(229, 231, 235, var(--tw-border-opacity)); border-style: solid; border-width: 0px; box-sizing: border-box; line-height: 30px; margin: 0px 0px 1.5rem;"><span style="font-family: inherit;">Each team that works closely together is frequently fine-tuning their meetings and work practices to make sure they’re a good fit for everyone in the team. A team environment needs structure but also adaptability and we try to promote both. Our style of working, as mentioned above, highlights our core values around flexibility. In the future, things may change and our working practices may require further adjusting both as employee headcount grows but also as people’s expectations and wants/needs change. As a bold, ambitious and inclusive company, Orbital Witness aims to keep ahead of this curve by continuously speaking with current and future employees and adapting how we work best, while always putting first how we can maximise the value we deliver to our customers.</span></p><p style="--tw-border-opacity: 1; --tw-ring-offset-shadow: 0 0 #0000; --tw-ring-shadow: 0 0 #0000; --tw-shadow: 0 0 #0000; border-color: rgba(229, 231, 235, var(--tw-border-opacity)); border-style: solid; border-width: 0px; box-sizing: border-box; line-height: 30px; margin: 0px 0px 1.5rem;"></p><div class="separator" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em; text-align: center;"><a href="https://blogger.googleusercontent.com/img/a/AVvXsEjFdlh12GafntblDOB7u6FgZjmQjAS59ahAOlT-KX6GoSJTCEml_piA1V4L6su61X9pwicIfQQDHeyoe7DuCVONu5CT05tg7t4wMYW_mnum6RcBvz3YFzx0EigGTRb8ko68W28kuesjrg-F5o78NDFHd7gWP74SEtf9bkLupb3dolAZXZBfq95R3W8qYA" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><span style="color: black;"><img alt="" data-original-height="666" data-original-width="1000" height="426" src="https://blogger.googleusercontent.com/img/a/AVvXsEjFdlh12GafntblDOB7u6FgZjmQjAS59ahAOlT-KX6GoSJTCEml_piA1V4L6su61X9pwicIfQQDHeyoe7DuCVONu5CT05tg7t4wMYW_mnum6RcBvz3YFzx0EigGTRb8ko68W28kuesjrg-F5o78NDFHd7gWP74SEtf9bkLupb3dolAZXZBfq95R3W8qYA=w640-h426" width="640" /></span></a></div><span style="font-family: inherit;"><br /><br /></span><p></p></div><br /></div>Andrewhttp://www.blogger.com/profile/03252749023463514361noreply@blogger.com0London, UK51.5072178 -0.127586223.196983963821154 -35.2838362 79.817451636178845 35.0286638tag:blogger.com,1999:blog-1847717784907730054.post-79019346925789372432022-01-11T23:45:00.010+00:002023-06-21T21:18:33.835+01:00Building software that 'thinks' like a lawyer<h3 style="text-align: justify;"><span style="text-align: left;">Codifying real estate legal expertise using data science and software engineering</span></h3><p style="text-align: justify;">This is a <a href="https://tech.orbitalwitness.com/posts/2021-10-29-build-software-that-thinks-like-a-lawyer/">repost</a> from <a href="https://tech.orbitalwitness.com/">Orbital Witness's Tech Blog</a> <span style="text-align: left;">that I recently setup with the engineering and data science teams to both recognise the work colleagues were doing internally as well as promote their work externally to attract potential candidates when hiring. At Orbital Witness we're building Natural Language Processing (NLP) systems for the real estate legal industry.</span></p><br /><hr /><div style="text-align: justify;"><br /></div><h2 style="text-align: justify;"><a href="https://lh3.googleusercontent.com/-vVB-G2PDjT8/Yd4PP08r80I/AAAAAAAA_o8/D9ZHU44eUfgeZrl1d7jbNqVpbHGxOx7ogCNcBGAsYHQ/image.png" style="clear: left; display: inline; margin-bottom: 1em; margin-right: 1em; text-align: left;"><img alt="" data-original-height="1616" data-original-width="2880" height="360" src="https://lh3.googleusercontent.com/-vVB-G2PDjT8/Yd4PP08r80I/AAAAAAAA_o8/D9ZHU44eUfgeZrl1d7jbNqVpbHGxOx7ogCNcBGAsYHQ/w640-h360/image.png" width="640" /></a></h2><div><div><h2 style="text-align: justify;"><br /></h2><h2 style="text-align: justify;">Presentation Video (9min)</h2></div><div style="text-align: justify;"><br /></div><div style="text-align: justify;">On October 5th, 2021 <a href="https://www.linkedin.com/in/andrewmatthewthompson/">Andrew Thompson (CTO)</a> and <a href="https://www.linkedin.com/in/brian-kennedy-lawyer">Brian Kennedy (Head of Legal Engineering & Innovation)</a> gave a presentation at the <a href="https://www.legalgeek.co/">LegalGeek</a> conference in Brick Lane, London. The title of their presentation was <b><i>Building software that ‘thinks’ like a lawyer</i></b>. It was a short 9 minute presentation where we highlighted five hurdles that arise when building such a system.</div><div style="text-align: justify;"><br /></div><div style="text-align: justify;">Lawyers, around the UK and abroad, currently conduct due diligence for their clients every day so that land and property can be bought, sold and developed. One of the main limiting aspects of this diligence process is the time taken to read through all legal documents to identify potential risks that could delay or halt a transaction. At Orbital Witness we’re solving this challenging problem by combining real estate legal knowledge, data science (specifically OCR - Optical Character Recognition and NLP - Natural Language Processing) and software engineering to automate real estate due diligence.</div><div style="text-align: justify;"><br /></div><br /></div>
<div style="text-align: center;">
<iframe allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture" allowfullscreen="" frameborder="0" height="360" src="https://www.youtube.com/embed/XjgIqdXnivQ" title="YouTube video player" width="640"></iframe></div><div style="text-align: left;"><h2 style="text-align: left;"><br /></h2><h2 style="text-align: left;">Document OCR</h2><div><br /></div><div>To pick up on just one hurdle mentioned in the above presentation, Hurdle #1 was concerned with how unpredictable OCR can be on legal real estate documents that Orbital Witness consumes. As can be seen in the image below, leases are available in varying degrees of quality and can often contain a variety of interesting artefacts such as:</div><div><br /></div><div><ul style="text-align: left;"><li>Both handwritten and typed textual content</li><li>Stamps that obscure textual content</li><li>Redacted sections of textual content</li><li>Struck out sections of textual content</li></ul><div><br /></div><div><div class="separator" style="clear: both; text-align: center;"><a href="https://lh3.googleusercontent.com/-A_RD4Z0omok/Yd4TK8lM_BI/AAAAAAAA_pE/fB3u0Gyq39gRlf-PhhSIaiTR1jved6S6gCNcBGAsYHQ/image.png" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img alt="" data-original-height="1620" data-original-width="2880" height="360" src="https://lh3.googleusercontent.com/-A_RD4Z0omok/Yd4TK8lM_BI/AAAAAAAA_pE/fB3u0Gyq39gRlf-PhhSIaiTR1jved6S6gCNcBGAsYHQ/w640-h360/image.png" width="640" /></a></div><br /><div><br /></div><div><br /></div><div><br /></div><div><br /></div><div><br /></div><div><br /></div><div><br /></div><div><br /></div><div><br /></div><div><br /></div><div><br /></div><div><br /></div><div><br /></div><div><br /></div><div><br /></div><div><br /></div><div><br /></div><div><br /></div><div><br /></div><div><br /></div><div><br /></div><div>Solving for these types of artefacts is one challenging step in a larger data pipeline. That pipeline starts with transporting and processing a collection of photocopied real estate documents and ends with real estate risks that have been accurately identified and classified according to the context of a transaction. This is no small feat to solve, from a data science and software engineering perspective, but we’re building a very capable team of lawyers, product managers, data scientists and software engineers to own this and other challenges and deliver solutions for them.</div><h2 style="text-align: left;"><br /></h2><h2 style="text-align: left;">We’re hiring</h2><div><br /></div><div>If you are interested in the above challenges and are curious to know more about how we currently solve them, please see our open roles and get in touch with us via our <a href="https://www.orbitalwitness.com/careers">Careers Page</a>. If nothing quite matches your experience then still feel free to connect and message me, <a href="https://www.linkedin.com/in/andrewmatthewthompson">Andrew Thompson</a>, directly on LinkedIn and I’d be happy to have a casual chat via video conference or over coffee ☕️.</div></div></div></div>Andrewhttp://www.blogger.com/profile/03252749023463514361noreply@blogger.com0London, UK51.5072178 -0.127586223.196983963821154 -35.2838362 79.817451636178845 35.0286638tag:blogger.com,1999:blog-1847717784907730054.post-42272848564269042472020-04-09T13:27:00.007+01:002022-01-12T00:23:09.203+00:00Simple Template For Tracking OKRs Weekly And Quarterly<div dir="ltr" style="text-align: left;" trbidi="on">
<div style="text-align: justify;">
Objectives and Key Results (or more commonly knowns as OKRs) are a well-defined way to codify a company's strategy along with tracking progress towards the intended results. John Doerr introduced OKRs at Google to great effect and has since written an insightful book called <a href="https://www.amazon.co.uk/Measure-What-Matters-Simple-Drives/dp/024134848X">Measure What Matters</a> that goes into the OKR framework and how it's successfully being used at various companies.</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
The OKR process can become overly complex rife with over analysing the data behind the numbers, poorly quantifying the key results and/or poorly communicating the ongoing impact they're having. I have seen many companies abandon the initiative part way through a quarter or even after a few quarters of failed attempts. </div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
I've distilled the OKR process for product engineering teams and the wider company down to a simple tracker that can be used on a weekly basis. Below I will explain the "team tracker" along with some examples, followed by how each team's OKRs come together into a simple view across the company that anyone can understand.</div>
<div style="text-align: justify;">
<br /></div>
<h2 style="text-align: justify;">
</h2>
<h2 style="text-align: justify;">
Team OKR Tracker</h2>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
The team OKR tracker is the starting point for the manager of each team. OKRs don't have a mandated length of time but I find doing them on a quarterly cadence allows enough time to make the upfront planning worth the time invested while still allowing for regular points throughout the year where strategic initiatives can be adjusted. Here is what a fresh version of the template looks like prior to a team's manager populating it with objectives, key results, owners and their associated weekly progress tracking:</div>
<div>
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://1.bp.blogspot.com/-bbQ40s-WpOY/XpjMqsIespI/AAAAAAAAny0/gphERfj5CiQ6MW9WhLQbosJ9fd3qhLjPwCLcBGAsYHQ/s1600/Blank%2BTeam%2BOKRs.png" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" data-original-height="542" data-original-width="1600" height="216" src="https://1.bp.blogspot.com/-bbQ40s-WpOY/XpjMqsIespI/AAAAAAAAny0/gphERfj5CiQ6MW9WhLQbosJ9fd3qhLjPwCLcBGAsYHQ/s640/Blank%2BTeam%2BOKRs.png" width="640" /></a></div>
<span style="text-align: justify;"><br /></span>
<span style="text-align: justify;">There are several parts of the above sheet template which are worth digging into further:</span><br />
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
</div>
<ul>
<li><b>Objective:</b> This is what you want to accomplish. An objective should be aspirational, significant to the company, personally meaningful and ambitious. I'd recommend having no more than 5 but ideally 3 or less objectives per team (the old adage of less is more can't be said enough when setting objectives).</li>
</ul>
<ul>
<li><b>Key Result: </b>This is how you will track the achievement of the objective. Key results should be easily measurable and limited in number. This is how the company measures progress towards its objectives while remaining flexible on the exact tactics to implement.</li>
</ul>
<ul>
<li><b>Owner: </b>There should be a single owner for any given key result who is responsible for driving its success. In a product team this is often the CTO, Head of Product/Engineering/Design, Product Owner, Scrum Master, Tech Lead, etc.</li>
</ul>
<ul>
<li><b>Weekly Percentage Tracking: </b>Each week throughout the quarter the owners of the OKRs should get together to discuss the progress of each one based on the percentage in the spreadsheet. Similar to a daily standup within a product team, any blockers or impediments should be raised and decisions made to resolve them. </li>
</ul>
<ul>
<li><b>Final: </b>This is simply the final result of each OKR at the end of the quarter. Note that sometimes it may take a week or two in order to determine the steady state of actual metrics associated with a given OKR (e.g. waiting for A/B test results for a feature deployed on the last day of the quarter).</li>
</ul>
<ul>
<li><b>Forecast: </b>This should be regularly discussed and re-forecasted if necessary to get an accurate picture of where the progress toward an OKR might end up at the end of a quarter. This is obviously hard to achieve with complete certainty but even when accounting for the usual margin of error, it's still a valuable way to indicate to everyone which OKRs are on track and which ones are struggling.</li>
</ul>
<ul>
<li><b>Notes: </b>Along with updating the weekly percentage of the OKRs, each owner can add a few notes about what is happening. This can be particularly valuable when a key result is not moving (aka still at 0%) but the team is making good progress with the build which will be deployed in the future and which will subsequently increase the key result pecentage.</li>
</ul>
<ul>
<li><b>Averages: </b>This is simply an average across all the OKRs for the team. If teams are aiming for the usual ~70% achievement rate (aka the OKRs taken together were ambitious but not impossible) this section highlights how the team is tracking on that overall goal.</li>
</ul>
<div>
<br /></div>
<h2 style="text-align: justify;">
</h2>
<h2 style="text-align: justify;">
Team OKR Tracker Examples</h2>
<div>
<br /></div>
<div>
The team OKR tracker below has been populated with examples of potential objectives, key results, owners and associated weekly progress tracking for a fictional product engineering team:<br />
<br /></div>
<div>
<div>
<a href="https://1.bp.blogspot.com/-CiWj0tNf6XY/XpjqvuuADnI/AAAAAAAAnzU/YnAJD2GMxm4Z4wptF_0qNu8p2XAtEIWSACPcBGAYYCw/s1600/Example%2BTeam%2BOKRS.png" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" data-original-height="542" data-original-width="1600" height="216" src="https://1.bp.blogspot.com/-CiWj0tNf6XY/XpjqvuuADnI/AAAAAAAAnzU/YnAJD2GMxm4Z4wptF_0qNu8p2XAtEIWSACPcBGAYYCw/s1600/Example%2BTeam%2BOKRS.png" width="640" /></a></div>
<div>
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br /></div>
<h2>
</h2>
<h2>
</h2>
<h2>
</h2>
<h2>
Company "Roll Up" OKR Tracker</h2>
</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
Each team within the company (e.g. Sales, Product, Marketing, People, etc) has their own OKR tracking sheet which is then "rolled up" into an overall high-level view. This view shows progress across the company against the previously-defined OKRs. It is able to give the executive team, the board and/or the rest of the company (via a monthly all hands meeting or weekly email) a simple view of which teams are doing well and which teams may need some additional help or resources in order to achieve their goals before the OKR period comes to an end.</div>
<div style="text-align: justify;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://1.bp.blogspot.com/--jFQp64hSC8/XpjNXyZqBdI/AAAAAAAAnzA/L7KsD9_rl3s_z5UjPtIgnwKEaGcKcyvQQCLcBGAsYHQ/s1600/Blank%2BCompany%2BOKRs.png" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" data-original-height="350" data-original-width="1600" height="140" src="https://1.bp.blogspot.com/--jFQp64hSC8/XpjNXyZqBdI/AAAAAAAAnzA/L7KsD9_rl3s_z5UjPtIgnwKEaGcKcyvQQCLcBGAsYHQ/s640/Blank%2BCompany%2BOKRs.png" width="640" /></a></div>
<div>
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<h2 style="text-align: left;">
Google Sheet Template</h2>
</div>
<br />
I've provided the OKR tracker as a Google Sheet which can be easily copied and used within your own company: <a href="https://docs.google.com/spreadsheets/d/1aEpVezdaaxrDv7hxex25ozS6QD_deP5cUcoKLn35aww">https://docs.google.com/spreadsheets/d/1aEpVezdaaxrDv7hxex25ozS6QD_deP5cUcoKLn35aww</a><br />
<br />
<iframe height="420" src="https://docs.google.com/spreadsheets/d/e/2PACX-1vQc5GnrVrokUCT10BvR7ElmEJJ3hgJncr5herBg7BlLDhIYcH1rKRsw-buHy4-H575M5mrN-Tf9P2OO/pubhtml?widget=true&headers=false" width="660"></iframe>
</div>
Andrewhttp://www.blogger.com/profile/03252749023463514361noreply@blogger.com0tag:blogger.com,1999:blog-1847717784907730054.post-35438840320887359792020-02-04T20:59:00.002+00:002022-01-12T00:00:14.220+00:00Interviewing Benedict Evans at Appear Here's Global Gathering<div dir="ltr" style="text-align: left;" trbidi="on">
<h2 style="text-align: left;">
The Venue and Agenda</h2>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://1.bp.blogspot.com/-1MrWn6_Lz1M/XjiXzMqQixI/AAAAAAAAlsc/dk5zwkqW9F03935JWdldwK7NAFq_ZU7TwCLcBGAsYHQ/s1600/83107269_111030853778647_855706019677181664_n.jpg" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1311" data-original-width="1080" height="320" src="https://1.bp.blogspot.com/-1MrWn6_Lz1M/XjiXzMqQixI/AAAAAAAAlsc/dk5zwkqW9F03935JWdldwK7NAFq_ZU7TwCLcBGAsYHQ/s320/83107269_111030853778647_855706019677181664_n.jpg" width="261" /></a><a href="https://1.bp.blogspot.com/-izz0Wg3XKZE/XjiX1_V1UfI/AAAAAAAAlsg/AEZji2ubF0kjUEP2xcyOtJtTZZwWihouQCLcBGAsYHQ/s1600/83857032_166939964588645_2324714873974385366_n.jpg" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1311" data-original-width="1080" height="320" src="https://1.bp.blogspot.com/-izz0Wg3XKZE/XjiX1_V1UfI/AAAAAAAAlsg/AEZji2ubF0kjUEP2xcyOtJtTZZwWihouQCLcBGAsYHQ/s320/83857032_166939964588645_2324714873974385366_n.jpg" width="261" /></a></div>
<br />
<div style="text-align: justify;">
On January 24th, 2020 Appear Here held its Annual Global Gathering where we brought together world class investors, retailers, landlords and technologists for a day of insights, lessons and knowledge sharing:</div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://1.bp.blogspot.com/-uGEA2_5GBo4/XjiojRURk3I/AAAAAAAAluQ/2vQD1SPSlIwwjToP--u6tWKwvg0oD6_bgCLcBGAsYHQ/s1600/global-gathering.png" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1430" data-original-width="1058" height="320" src="https://1.bp.blogspot.com/-uGEA2_5GBo4/XjiojRURk3I/AAAAAAAAluQ/2vQD1SPSlIwwjToP--u6tWKwvg0oD6_bgCLcBGAsYHQ/s320/global-gathering.png" width="236" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<h2 style="clear: both; text-align: justify;">
My chat with Benedict Evans</h2>
<div class="separator" style="clear: both; text-align: justify;">
<br /></div>
<div class="separator" style="clear: both; text-align: justify;">
I had the pleasure of hosting a fireside chat with the infamous Benedict Evans (formerly a Partner at <a href="https://a16z.com/">a16z</a> and now based in London). Benedict is w<span style="text-align: left;">idely considered to be a thought leader for the tech industry and his <a href="https://www.ben-evans.com/">weekly newsletter</a> frequently shapes the conversation around new tech developments, artificial intelligence, regulation, big tech companies, mobile and venture capital for a global audience. He has an avid following which includes 130k newsletter subscribers and 280k Twitter followers.</span></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://1.bp.blogspot.com/-qrr2SwJbaEA/XjiZwvEaQyI/AAAAAAAAltA/JD6rnWLZD_U96JzN5S_oo8D0YqadfwS8gCLcBGAsYHQ/s1600/Global%2BGathering%2B2020%2BTeam%2B340.jpg" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" data-original-height="933" data-original-width="1400" height="426" src="https://1.bp.blogspot.com/-qrr2SwJbaEA/XjiZwvEaQyI/AAAAAAAAltA/JD6rnWLZD_U96JzN5S_oo8D0YqadfwS8gCLcBGAsYHQ/s640/Global%2BGathering%2B2020%2BTeam%2B340.jpg" width="640" /></a></div>
<div class="separator" style="clear: both; text-align: justify;">
<br /></div>
<div class="separator" style="clear: both; text-align: justify;">
We covered a range of topics including high street retail, machine learning/data science and acquisition channels for brands. Below I've included the list of questions we worked our way through during the 30min conversation:</div>
<div style="text-align: justify;">
<br /></div>
<div class="separator" style="clear: both; text-align: justify;">
1. You've said. "Everything the Internet has done to media, it's now doing to retail", tell us more.</div>
<div class="separator" style="clear: both; text-align: justify;">
<br /></div>
<div class="separator" style="clear: both; text-align: justify;">
2. Do you think the high street is dying? Why or why not?</div>
<div class="separator" style="clear: both; text-align: justify;">
<br /></div>
<div class="separator" style="clear: both; text-align: justify;">
3. What does the future for the high street look like?</div>
<div class="separator" style="clear: both; text-align: justify;">
<br /></div>
<div class="separator" style="clear: both; text-align: justify;">
4. If it was up to you, in what ways would you like to see retail change or be disrupted by tech?</div>
<div class="separator" style="clear: both; text-align: justify;">
<br /></div>
<div class="separator" style="clear: both; text-align: justify;">
5. What role do you think machine learning/data science could play in retail?</div>
<div class="separator" style="clear: both; text-align: justify;">
<br /></div>
<div class="separator" style="clear: both; text-align: justify;">
6. We hear a lot about machine learning and data science, what are some useful ways to think about machine learning? (You've said, "Machine learning is the new database" tell us more)</div>
<div class="separator" style="clear: both; text-align: justify;">
<br /></div>
<div class="separator" style="clear: both; text-align: justify;">
7. As the cost of online advertising increases, how do marketers continue to grow their businesses?</div>
<div class="separator" style="clear: both; text-align: justify;">
<br /></div>
<div class="separator" style="clear: both; text-align: justify;">
8. Why do you think Amazon hasn't been as successful with offline retail as they are with online?</div>
<div class="separator" style="clear: both; text-align: justify;">
<br /></div>
<div class="separator" style="clear: both; text-align: justify;">
9. What do you make of the rise and fall of WeWork; what can we learn from it?</div>
<div class="separator" style="clear: both; text-align: justify;">
<br /></div>
<div class="separator" style="clear: both; text-align: justify;">
10. After working in SF and London, what differences do you see and what can we learn from them?</div>
<div class="separator" style="clear: both; text-align: justify;">
<br /></div>
<div class="separator" style="clear: both; text-align: justify;">
<br /></div>
<h2 style="clear: both; text-align: justify;">
Benedict's "Standing on the Shoulders of Giants" Presentation from Davos</h2>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
Just earlier that week, Benedict had been in Davos to present an annual presentation on tech trends. In a recent blog post he said "Every year, I produce a big presentation digging into macro and strategic trends in the tech industry. This year, ‘Standing on the shoulders of giants’ looks at what it means that 4bn people have a smartphone; we connected everyone, and now we wonder what the Next Big Thing is, but meanwhile, connecting everyone means we connected all the problems. Tech is becoming a regulated industry, but we don’t really know what that will mean." I've pulled out some notable slides from that <a href="https://www.ben-evans.com/presentations">presentation</a> below that relate to the discussion we had at Appear Here's Global Gathering:</div>
<div style="text-align: justify;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://1.bp.blogspot.com/-Jfv23y0-XHU/XjijfUCeobI/AAAAAAAAltQ/K6Bk42q04_0OOUz93U2nvym5XJBT5t4jwCLcBGAsYHQ/s1600/slide15.png" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" data-original-height="880" data-original-width="1600" height="352" src="https://1.bp.blogspot.com/-Jfv23y0-XHU/XjijfUCeobI/AAAAAAAAltQ/K6Bk42q04_0OOUz93U2nvym5XJBT5t4jwCLcBGAsYHQ/s640/slide15.png" width="640" /></a></div>
<div style="text-align: justify;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://1.bp.blogspot.com/-p9pwvaKnyaE/Xjij2MCnVGI/AAAAAAAAltY/u4viVBqREfwR8W7mF6zDMdErO2YwzYyPACLcBGAsYHQ/s1600/slide16.png" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" data-original-height="877" data-original-width="1600" height="350" src="https://1.bp.blogspot.com/-p9pwvaKnyaE/Xjij2MCnVGI/AAAAAAAAltY/u4viVBqREfwR8W7mF6zDMdErO2YwzYyPACLcBGAsYHQ/s640/slide16.png" width="640" /></a></div>
<div style="text-align: justify;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://1.bp.blogspot.com/-2SR0sCjni5g/Xjik9U2ua9I/AAAAAAAAltk/bYQyVmyDsGALmCF760S-v6iFIMeSwIKggCLcBGAsYHQ/s1600/slide18.png" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" data-original-height="875" data-original-width="1600" height="350" src="https://1.bp.blogspot.com/-2SR0sCjni5g/Xjik9U2ua9I/AAAAAAAAltk/bYQyVmyDsGALmCF760S-v6iFIMeSwIKggCLcBGAsYHQ/s640/slide18.png" width="640" /></a></div>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://1.bp.blogspot.com/-3vJIICkim_U/Xjik-yc2srI/AAAAAAAAlts/QrYxdGIu_fgHlhaP-nrSEtC2Ot2wA3g2wCLcBGAsYHQ/s1600/slide25.png" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" data-original-height="878" data-original-width="1600" height="350" src="https://1.bp.blogspot.com/-3vJIICkim_U/Xjik-yc2srI/AAAAAAAAlts/QrYxdGIu_fgHlhaP-nrSEtC2Ot2wA3g2wCLcBGAsYHQ/s640/slide25.png" width="640" /></a></div>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://1.bp.blogspot.com/-oQkOQ-hi50c/Xjik-2Yka8I/AAAAAAAAlto/0PPZdxCFKMI3JZmbFTg4Ou0uSXvqiXpLgCLcBGAsYHQ/s1600/slide27.png" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" data-original-height="880" data-original-width="1600" height="352" src="https://1.bp.blogspot.com/-oQkOQ-hi50c/Xjik-2Yka8I/AAAAAAAAlto/0PPZdxCFKMI3JZmbFTg4Ou0uSXvqiXpLgCLcBGAsYHQ/s640/slide27.png" width="640" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://1.bp.blogspot.com/-ktgAxxitCa4/Xjil7ngqIbI/AAAAAAAAluA/zgywjMOHIU4r41QFYAn1RPRCLOCsVjtFACLcBGAsYHQ/s1600/slide31.png" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" data-original-height="875" data-original-width="1600" height="350" src="https://1.bp.blogspot.com/-ktgAxxitCa4/Xjil7ngqIbI/AAAAAAAAluA/zgywjMOHIU4r41QFYAn1RPRCLOCsVjtFACLcBGAsYHQ/s640/slide31.png" width="640" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://1.bp.blogspot.com/-T4ERPifhJKM/Xjil7RVdoWI/AAAAAAAAlt8/yGAC8TDbfTAmSayVGr0x_6QqINyC2aCwwCLcBGAsYHQ/s1600/slide33.png" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em; text-align: justify;"><img border="0" data-original-height="877" data-original-width="1600" height="350" src="https://1.bp.blogspot.com/-T4ERPifhJKM/Xjil7RVdoWI/AAAAAAAAlt8/yGAC8TDbfTAmSayVGr0x_6QqINyC2aCwwCLcBGAsYHQ/s640/slide33.png" width="640" /></a><span style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em; text-align: right;"><span style="text-align: justify;"><br /></span></span><span style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em; text-align: left;"><span style="text-align: justify;">Benedict has clearly thought carefully about how technology is evolving and it was a pleasure to speak with him about those trends as they apply to retail, data science and startups.</span></span></div>
</div>
Andrewhttp://www.blogger.com/profile/03252749023463514361noreply@blogger.com0tag:blogger.com,1999:blog-1847717784907730054.post-59227328759660157242019-10-02T16:34:00.000+01:002019-11-08T12:56:33.571+00:00Speaking at Movers & Shakers Property Networking Forum<div dir="ltr" style="text-align: left;" trbidi="on">
<div style="text-align: justify;">
I'll be speaking at the Movers & Shakers Property Networking Forum on Nov 8th, 2019 on "Is Technology driving a more agile & fluid real estate market". I'll be accompanied by a panel consisting of Guy Grainger (Chief Executive EMEA) at JLL, Emma Long (Head of Corporate Strategy) at Legal & General Real Assets, Roelof Opperman (Principal and Co-lead) at Fifth Wall and Dr Rick Robinson (Digital Property and Cities Leader) at Arup along with 200 attendees from the property, startup and investment sector.</div>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://1.bp.blogspot.com/-8IK2XJWXowU/XcVTFVkC32I/AAAAAAAAjyw/WhiUH5teZVcZ571gBvOpROwzqrtUVVMTACLcBGAsYHQ/s1600/movers-and-shakers-speaking-panel.png" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" data-original-height="1251" data-original-width="1600" height="500" src="https://1.bp.blogspot.com/-8IK2XJWXowU/XcVTFVkC32I/AAAAAAAAjyw/WhiUH5teZVcZ571gBvOpROwzqrtUVVMTACLcBGAsYHQ/s640/movers-and-shakers-speaking-panel.png" width="640" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://www.moversandshakers.uk.com/events/detail/1179"><span style="font-size: x-small;">https://www.moversandshakers.uk.com/events/detail/1179</span></a></div>
<div style="text-align: center;">
<br /></div>
</div>
Andrewhttp://www.blogger.com/profile/03252749023463514361noreply@blogger.com0tag:blogger.com,1999:blog-1847717784907730054.post-7778661220813532552019-04-05T09:47:00.000+01:002019-11-08T13:36:27.190+00:00Speaking at LDC's 19th Retail and Leisure Summit Event<div dir="ltr" style="text-align: left;" trbidi="on">
<span style="text-align: justify;">I'll be speaking at LDC's 19th Retail and Leisure Summit Event on May 1st about "Retail Done Differently". I'll be discussing Appear Here and how we've created a new business model in commercial retail that uses technology to match ideas to physical retail space in innovate ways.</span><br />
<span style="text-align: justify;"><br /></span>
<a href="https://1.bp.blogspot.com/-62fkc1AxaIg/XcVgvuNzkiI/AAAAAAAAjzU/Y6QRBQfVYEA-LQSS0es123qEMgsNF6noACLcBGAsYHQ/s1600/ldc-bdo-event-may-1st-speakers-only.png" imageanchor="1" style="clear: left; display: inline !important; margin-bottom: 1em; margin-right: 1em; text-align: center;"><img border="0" data-original-height="1421" data-original-width="1600" height="568" src="https://1.bp.blogspot.com/-62fkc1AxaIg/XcVgvuNzkiI/AAAAAAAAjzU/Y6QRBQfVYEA-LQSS0es123qEMgsNF6noACLcBGAsYHQ/s640/ldc-bdo-event-may-1st-speakers-only.png" width="640" /></a><br />
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://1.bp.blogspot.com/-8pEekU9N9Mc/XcVgvSTKdnI/AAAAAAAAjzQ/8nPFCE6cNfMeUWmQZFPUkqE96hqzBp0iwCLcBGAsYHQ/s1600/ldc-bdo-event-may-1st-agenda-only.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1600" data-original-width="1457" height="640" src="https://1.bp.blogspot.com/-8pEekU9N9Mc/XcVgvSTKdnI/AAAAAAAAjzQ/8nPFCE6cNfMeUWmQZFPUkqE96hqzBp0iwCLcBGAsYHQ/s640/ldc-bdo-event-may-1st-agenda-only.png" width="582" /></a></div>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://www.localdatacompany.com/ldcs-19th-retail-and-leisure-trends-summit"><span style="font-size: x-small;">https://www.localdatacompany.com/ldcs-19th-retail-and-leisure-trends-summit</span></a></div>
</div>
Andrewhttp://www.blogger.com/profile/03252749023463514361noreply@blogger.com0tag:blogger.com,1999:blog-1847717784907730054.post-92220547882154342542019-01-03T14:52:00.000+00:002019-01-04T12:25:57.924+00:00How We Hire Product Developers at Appear Here<div dir="ltr" style="text-align: left;" trbidi="on">
I recently wrote <a href="https://medium.com/appear-here-product-engineering/how-we-hire-product-developers-at-appear-here-9a2a684b1b08">this post</a> on Appear Here's Product Engineering blog so potential developer candidates, that I wanted to hire, would better understand our hiring process. I've reposted it here.<br />
<br />
<hr />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://4.bp.blogspot.com/-ZfBEdwhKnU0/XC9NDWeUCUI/AAAAAAAAWgk/Bsxw3RnBZDMyUqQVTVtxHaw1_o0E1rrcACLcBGAs/s1600/Appear%2BHere%2BDeveloper%2BRecruitment%2BProcess.jpg" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" data-original-height="1001" data-original-width="1600" height="400" src="https://4.bp.blogspot.com/-ZfBEdwhKnU0/XC9NDWeUCUI/AAAAAAAAWgk/Bsxw3RnBZDMyUqQVTVtxHaw1_o0E1rrcACLcBGAs/s640/Appear%2BHere%2BDeveloper%2BRecruitment%2BProcess.jpg" width="640" /></a></div>
<br />
<div style="clear: both; text-align: center;">
<span style="font-size: x-small;">The three steps for hiring developers which takes approximately one week start to finish</span></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
At Appear Here, our mission is to create a world where anyone with an idea can find space to make it happen. Our Product team is using the latest technologies, along with applying machine learning in an innovative way, to solve challenging problems that affect retail today.<br />
<br />
When hiring developers for our Product team at Appear Here, we aim to find the right balance between (1) giving you the necessary information about us and what we do so that you can make an informed decision about your next career move, (2) evaluating your technical skills and cultural fit and (3) appreciating that you’re most likely interviewing at a handful of companies which can take up a substantial amount of time during and outside of your current day job.<br />
<br />
We like to move quickly so we’ve created this three step process that typically takes one week start to finish (i.e. from your CV submission to sending you an offer).<br />
<br />
<h2 style="text-align: left;">
<b>Step One: Initial 30 Minute Phone Call</b></h2>
<br />
The purpose of this call is to accomplish the following:<br />
<br />
<ol style="text-align: left;">
<li>Give you a high-level overview of what we do as a company (i.e. why we exist, our mission and the burning problem are we solving in the world)</li>
<li>Describe how we currently achieve our mission as a company and specifically a Product team of developers, designers, data scientists and product managers (i.e. our structure and size, our current development processes and technologies, and our plans for the upcoming year)</li>
<li>Gauge your interest in what we’re achieving as a company along with assessing your communication skills</li>
<li>Answer any questions you might have</li>
</ol>
<br />
To prepare for the call I’d recommend reading <a href="https://www.theguardian.com/small-business-network/2017/jul/06/airbnb-for-retail-high-street-business-rates-26-grains-littlesmith-appear-here">this Guardian article</a> which referred to us as the ”Airbnb for Retail”, watching <a href="https://www.youtube.com/watch?v=VUBByt6EuUM">this 5 minute Sky News broadcast</a> with Appear Here’s CEO talking about Saving Retail and, if you’re curious about the underlying business models in this space, read <a href="https://medium.com/general-catalyst-amplified/the-future-of-offline-retail-288ad1b027cb">this article</a> titled “The Future of Offline Retail” by Addie Lerner of General Catalyst. Please also prepare a few insightful questions that would help you determine whether Appear Here is right for the next step in your career.<br />
<br />
The phone call is typically with myself, Andrew Thompson (I’m currently CTO of Appear Here) and I try to schedule it at a time that’s convenient for you during the work day.<br />
<br />
<h2 style="text-align: left;">
<b>Step Two: One Hour Live Coding Challenge via Video Conference</b></h2>
<br />
Next, we schedule a one hour video conference where you will be given a coding challenge by one of our developers depending on the specific skills you have across the stack (e.g. Javascript, React, Ruby on Rails, AWS, etc).<br />
<br />
The purpose of this coding challenge is not to assess your entire development experience and skill set in one hour but instead to get a basic understanding of your coding proficiency in an area of the development stack that you are most comfortable coding in. We’re specifically looking for you to demonstrate the following to us:<br />
<br />
<ul style="text-align: left;">
<li>Complete the task in a reasonable amount of time</li>
<li>Keep your solution simple and avoid unnecessary over-engineering</li>
<li>Understand new code relatively quickly</li>
<li>Read and follow instructions</li>
<li>Use sensible Javascript/React/Ruby functions or methods</li>
</ul>
<br />
Our coding challenges are publicly available on Github: <a href="https://github.com/appearhere/product-challenges/tree/master/challenges">https://github.com/appearhere/product-challenges/tree/master/challenges</a><br />
<br />
<h2 style="text-align: left;">
<b>Step Three: Half-Day Interview</b></h2>
<br />
After passing the coding challenge, we will schedule you for a half day of interviews where you will meet several members of the existing Product team (developers, designers, data scientists and product managers) along with a couple of members of our culture board who are from other departments at the company.<br />
<br />
You will be asked a variety of questions in each interview about your skills, experience and motivation for wanting to work at Appear Here. We’ll also spend time getting to know you a little bit throughout this process (e.g. what do you do on the weekend for fun, what excites you outside of work, etc).<br />
<br />
Although some developers care more about the underlying technologies, we’re specifically looking for developers who care first and foremost about solving the challenges our customers face. As a company we have a strong bias toward a “product first, technology second” approach. We first try to understand what product features our customers would benefit from and why, before thinking about what technologies we might use to implement them. It’s also important for us to find out whether you truly care about the mission we have as a company to make ideas happen.<br />
<br />
Once all feedback from the interviews has been received, we will review it and make a quick decision to send you an offer.<br />
<br />
<h2 style="text-align: left;">
<b>Company Values</b></h2>
<br />
Our engineers take ownership to tackle challenging problems, and we are looking for developers to join us that are bright, humble and passionate about transforming retail and commercial real estate. Appear Here has a strong sense of purpose and has, from the very beginning of the company’s existence, had a set of values that all employees strive to embody, as outlined below:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://1.bp.blogspot.com/-P-mEjUm_WlA/XC3w0utBABI/AAAAAAAAWfI/dryNwGmvtdc2d308OpCuFDCCizt1w8mNwCLcBGAs/s1600/appear%2Bhere%2Bvalues.png" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" data-original-height="1074" data-original-width="1600" height="427" src="https://1.bp.blogspot.com/-P-mEjUm_WlA/XC3w0utBABI/AAAAAAAAWfI/dryNwGmvtdc2d308OpCuFDCCizt1w8mNwCLcBGAs/s640/appear%2Bhere%2Bvalues.png" width="640" /></a></div>
<b><br /></b>
<b><br /></b><br />
<h2 style="text-align: left;">
<b>Open Roles</b></h2>
<br />
We currently have many roles open in our Product team including:<br />
<br />
<ul style="text-align: left;">
<li>Full Stack Engineer: <a href="https://appearhere.workable.com/jobs/426888">https://appearhere.workable.com/jobs/426888</a></li>
<li>Frontend Engineer: <a href="https://appearhere.workable.com/jobs/594360">https://appearhere.workable.com/jobs/594360</a></li>
<li>Backend Engineer: <a href="https://appearhere.workable.com/jobs/597956">https://appearhere.workable.com/jobs/597956</a></li>
<li>Product Manager: <a href="https://appearhere.workable.com/jobs/570227">https://appearhere.workable.com/jobs/570227</a></li>
</ul>
<br />
You can find more about other open roles and our company culture on the <a href="https://www.appearhere.co.uk/about/careers">Careers section of our website</a>. Submit your CV and I’ll be in touch shortly to set up a chat.</div>
Andrewhttp://www.blogger.com/profile/03252749023463514361noreply@blogger.com0tag:blogger.com,1999:blog-1847717784907730054.post-82901268574695841472019-01-02T21:05:00.001+00:002019-01-04T17:59:31.630+00:00Appear Here's Case Study with Hired<div dir="ltr" style="text-align: left;" trbidi="on">
<div class="separator" style="clear: both; text-align: justify;">
I recently worked with <a href="https://hired.com/">Hired</a>, the marketplace for tech talent, on a case study. I've been using their platform to grow the product engineering team at <a href="https://www.appearhere.co.uk/">Appear Here</a> and in the last year have hired numerous engineers through it. Here is the case study as a two page PDF:</div>
<div class="separator" style="clear: both; text-align: justify;">
<br /></div>
<div style="text-align: justify;">
<iframe height="820" src="https://drive.google.com/file/d/1gqMZG2EQgRYVMN-2ZeHpPx-3z23IJzAD/preview" width="660"></iframe>
</div>
</div>
Andrewhttp://www.blogger.com/profile/03252749023463514361noreply@blogger.com0tag:blogger.com,1999:blog-1847717784907730054.post-56433458824856593172018-09-01T21:37:00.000+01:002018-11-14T12:09:47.811+00:00Speaking at Facebook's Leadership Summit & Hired's Employer Brand Health Panel<div dir="ltr" style="text-align: left;" trbidi="on">
<div style="text-align: justify;">
On Wednesday, September 19th I'll be speaking at both of <a href="https://fbleadershipsummit2018.splashthat.com/">Facebook's Leadership Summit</a> and <a href="http://resources.hired.com/brandhealth_london">Hired's Employer Brand Health Panel</a>. </div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
At Facebook I'll be talking on a panel about "Retaining Culture" along with <a href="https://www.linkedin.com/in/gymtan">Gerald Tan</a> (VP Growth & Product at Catapult Sports), <a href="https://www.linkedin.com/in/mariazverina">Maria Zverina</a> (Engineering Manager at Facebook) and <a href="https://www.linkedin.com/in/jessie-link-2a6ab546">Jessie Link</a> (Senior Director of Engineering at Twitter). This panel will be apart of an all day conference based out of Facebook's 1 Rathbone Square offices in Fitzrovia and will begin at 11am.</div>
<div style="text-align: justify;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://3.bp.blogspot.com/-CBlYSa3xVlI/W6ANn7t6uII/AAAAAAAATk4/VtQXp-QzJRs9c7RoOSu7ykkqJSNEWk16wCLcBGAs/s1600/Facebook%2BLeadership%2BSummit.png" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" data-original-height="878" data-original-width="1600" height="348" src="https://3.bp.blogspot.com/-CBlYSa3xVlI/W6ANn7t6uII/AAAAAAAATk4/VtQXp-QzJRs9c7RoOSu7ykkqJSNEWk16wCLcBGAs/s640/Facebook%2BLeadership%2BSummit.png" width="640" /></a></div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
At Hired I'll be talking on a panel about "Employer Branding" along with <a href="https://www.linkedin.com/in/kreska/">Marta Jasinska</a> (CTO at Moo), <a href="https://www.linkedin.com/in/seanfanning1/">Sean Fanning</a> (Talent Manager at Improbable), <a href="https://www.linkedin.com/in/mm-campbell/">Maria Campbell</a> (VP People at Snyk) and moderated by <a href="https://www.linkedin.com/in/gosmithlinkedin/">Gordon Smith</a> (UK Sales Director at Hired). This panel will be hosted out of Nobu Hotel Shoreditch and will begin at 4:30pm.</div>
<div style="text-align: justify;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://2.bp.blogspot.com/-AStPQ1VIPPo/W6AQUtd6MAI/AAAAAAAATlE/trYy8WRZG5QdceeF5TyTwD74zA1cwzrcwCLcBGAs/s1600/Hired%2BEmployer%2BBranding%2BPanel.png" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" data-original-height="844" data-original-width="1600" height="336" src="https://2.bp.blogspot.com/-AStPQ1VIPPo/W6AQUtd6MAI/AAAAAAAATlE/trYy8WRZG5QdceeF5TyTwD74zA1cwzrcwCLcBGAs/s640/Hired%2BEmployer%2BBranding%2BPanel.png" width="640" /></a></div>
<div style="text-align: justify;">
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<div style="text-align: left;">
<br />
<div style="text-align: justify;">
<br />
<hr />
</div>
</div>
<div style="text-align: justify;">
<h2 style="clear: both;">
</h2>
<h2 style="clear: both;">
</h2>
<h2 style="clear: both;">
</h2>
<h2 style="clear: both;">
</h2>
<h2 style="clear: both;">
</h2>
<h2 style="clear: both;">
<br /></h2>
<h2 style="clear: both;">
Update</h2>
</div>
<br />
During the Hired panel a videographer was recording a few moments and Hired have compiled this one minute video of the event:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<iframe allowfullscreen='allowfullscreen' webkitallowfullscreen='webkitallowfullscreen' mozallowfullscreen='mozallowfullscreen' width='640' height='360' src='https://www.blogger.com/video.g?token=AD6v5dy19Ia6j9ibXm3crvCOeoXPU-PZiuPk4iKZQudU_e-5cgBWrzQbEdjUTHwM80Nwvr2RPlpyYpSc9q054NYpQQ' class='b-hbp-video b-uploaded' frameborder='0'></iframe></div>
</div>
</div>
Andrewhttp://www.blogger.com/profile/03252749023463514361noreply@blogger.com1tag:blogger.com,1999:blog-1847717784907730054.post-49890225368788635652017-08-19T18:30:00.001+01:002017-08-19T18:30:28.988+01:00Joining Appear Here as CTO<div dir="ltr" style="text-align: left;" trbidi="on">
<div style="text-align: justify;">
<div class="separator" style="clear: both; text-align: justify;">
<br /></div>
<h2 style="clear: both; text-align: left;">
<span style="text-align: justify;">Brand and Product</span></h2>
<div>
<span style="text-align: justify;"><br /></span></div>
</div>
<div style="text-align: justify;">
<div style="text-align: center;">
<div class="separator" style="clear: both; text-align: center;">
<a href="https://1.bp.blogspot.com/-indhBmCCeZs/WZhyFg3u8dI/AAAAAAAAK78/-Wf4_lh5_RAm1DBc9InyJDKLIyKQahlGwCLcBGAs/s1600/appear-here-crop.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em; text-align: center;"><img border="0" data-original-height="297" data-original-width="1111" height="106" src="https://1.bp.blogspot.com/-indhBmCCeZs/WZhyFg3u8dI/AAAAAAAAK78/-Wf4_lh5_RAm1DBc9InyJDKLIyKQahlGwCLcBGAs/s400/appear-here-crop.png" width="400" /></a></div>
<div style="text-align: justify;">
</div>
<div style="text-align: justify;">
<br /></div>
</div>
</div>
<div style="text-align: justify;">
<div style="text-align: center;">
<iframe allowfullscreen="" frameborder="0" height="360" src="https://www.youtube.com/embed/DWc43kQ2ZvQ?rel=0&showinfo=0" width="640"></iframe><br /></div>
<div style="text-align: justify;">
<br />
Appear Here's very clear and focused mission resonates with me because they want to create a world where anyone with an idea can find space to make it happen. They achieve this by bringing together a community of brands, entrepreneurs and creatives in the world's best cities. They already have 80,000+ global brands and creative entrepreneurs, 10,000,000+ sq ft of spaces and 3 offices globally. Brands and entrepreneurs can use the Appear Here platform to find and book short-term spaces such as high street boutiques, market, shopping centres and shop shares in London, Paris and New York.</div>
</div>
<div style="text-align: justify;">
<div style="text-align: center;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://3.bp.blogspot.com/-nBIaOkJt-YA/WZWyrQs49XI/AAAAAAAAK40/-vT05zXhHLYA6nAETxAmTG6_sJMNAU_EgCLcBGAs/s1600/fitzrovia-search.png" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em; text-align: center;"><img border="0" data-original-height="659" data-original-width="1276" height="329" src="https://3.bp.blogspot.com/-nBIaOkJt-YA/WZWyrQs49XI/AAAAAAAAK40/-vT05zXhHLYA6nAETxAmTG6_sJMNAU_EgCLcBGAs/s640/fitzrovia-search.png" width="640" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<h2>
Company Culture</h2>
<br />
From the very first interview to signing my contract and officially joining, the team at Appear Here have built a company that continuously upholds these values:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://4.bp.blogspot.com/-KtlYJFHXIjw/WY4vagAvT6I/AAAAAAAAK2U/hdZ8PsfEokwFAvtD9u8z4Bah5bDehIoQgCLcBGAs/s1600/appear-here-values.png" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" data-original-height="905" data-original-width="1600" height="360" src="https://4.bp.blogspot.com/-KtlYJFHXIjw/WY4vagAvT6I/AAAAAAAAK2U/hdZ8PsfEokwFAvtD9u8z4Bah5bDehIoQgCLcBGAs/s640/appear-here-values.png" width="640" /></a></div>
<div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
</div>
<h2>
We're Hiring</h2>
<div>
<br />
<a href="https://www.appearhere.co.uk/about/careers">We're hiring talented people</a> to help make this ambitious vision a reality. We're currently looking for <a href="https://appearhere.workable.com/jobs/426888?">Software Engineers</a> and we will soon be hiring for other product and engineering roles as we grow. If you're interested in joining me and the rest of the Appear Here team, feel free to reach out to me directly (via <a href="https://www.linkedin.com/in/andrewmatthewthompson">Linkedin</a> or <a href="https://twitter.com/andrewthompsonx">Twitter</a>) and we'll have a coffee to chat through what we're up to.</div>
</div>
<div style="text-align: justify;">
<div class="separator" style="clear: both; text-align: justify;">
<br /></div>
<div class="separator" style="clear: both; text-align: justify;">
I'm looking forward to taking on this new responsibility as Appear Here's CTO and working with the team to drive the business forward. </div>
<div class="separator" style="clear: both; text-align: justify;">
<br /></div>
<div class="separator" style="clear: both; text-align: justify;">
<br /></div>
<h2 style="clear: both; text-align: justify;">
Recent Press</h2>
<div>
<br /></div>
<div class="separator" style="clear: both; text-align: justify;">
</div>
<ul>
<li><a href="https://techcrunch.com/2017/05/17/appear-here-a-marketplace-for-short-term-retail-space-raises-12m-series-b">Appear Here, a marketplace for short-term retail space, raises £12M Series B</a></li>
<li><a href="https://www.theguardian.com/small-business-network/2017/jul/06/airbnb-for-retail-high-street-business-rates-26-grains-littlesmith-appear-here">How an 'Airbnb for retail' is bringing entrepreneurs back to the high-street</a></li>
<li><a href="http://www.thedrum.com/news/2017/02/22/appear-here-the-airbnb-retail-eyes-big-brand-budgets-with-launch-global-projects">Appear Here - 'the Airbnb for retail' - eyes big brand budgets with launch of Global Projects team</a></li>
</ul>
</div>
</div>
Andrewhttp://www.blogger.com/profile/03252749023463514361noreply@blogger.com0London, UK51.5073509 -0.1277582999999822351.1912379 -0.77320529999998222 51.8234639 0.51768870000001777tag:blogger.com,1999:blog-1847717784907730054.post-70601051462980082172017-07-12T01:39:00.000+01:002017-07-25T19:07:50.727+01:00Building a "Job Finder" skill for my Amazon Alexa Device<div dir="ltr" style="text-align: left;" trbidi="on">
<h2 style="clear: both; text-align: justify;">
Growth</h2>
<div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://4.bp.blogspot.com/-kFVxZaz7mc4/WWX4tEUAGnI/AAAAAAAAKnU/-FZ7AJrhhkUpbUWv_YBzYVAfB-Nb10BqgCLcBGAs/s1600/Dot_family-1240x698.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="698" data-original-width="1240" height="225" src="https://4.bp.blogspot.com/-kFVxZaz7mc4/WWX4tEUAGnI/AAAAAAAAKnU/-FZ7AJrhhkUpbUWv_YBzYVAfB-Nb10BqgCLcBGAs/s400/Dot_family-1240x698.jpg" width="400" /></a></div>
</div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div style="text-align: justify;">
</div>
<div class="separator" style="clear: both; text-align: justify;">
According to a recent <a href="https://twitter.com/andrewthompsonx/status/884375698420494336">NPR and Edison Research Smart Audio Report</a>, 5% of the 1,620 surveyed respondents owned an Alexa device (Echo or Echo Dot) and 2% owned a Google Home device. Even more revealing was the fact that 42% of Smart Speaker owners said that "their Smart Speakers are essential to their everyday lives". By any stretch, Smart Speakers have reached a significant level of product/market fit and in a little over 2 years Amazon Alexa devices are leading the pack and growing at a rapid pace:</div>
<div class="separator" style="clear: both; text-align: justify;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://2.bp.blogspot.com/-HUASdcaRD_c/WWWCM-SrUfI/AAAAAAAAKmw/MNJPrmKXjOojxI__oYfJxw5S3w3agCbvACLcBGAs/s1600/alexa-stats.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="900" data-original-width="1200" height="300" src="https://2.bp.blogspot.com/-HUASdcaRD_c/WWWCM-SrUfI/AAAAAAAAKmw/MNJPrmKXjOojxI__oYfJxw5S3w3agCbvACLcBGAs/s400/alexa-stats.jpg" width="400" /></a></div>
<div style="clear: both; text-align: center;">
<div class="separator" style="clear: both; text-align: center;">
<a href="http://uk.businessinsider.com/amazon-echo-sales-figures-stats-chart-2016-12"><span style="font-size: x-small;">Business Insider: Amazon Echo sales and stats - Dec 2016</span></a></div>
<br /></div>
<h2 style="clear: both; text-align: justify;">
Skill</h2>
<div class="separator" style="clear: both; text-align: justify;">
<br /></div>
<div class="separator" style="clear: both; text-align: justify;">
I decided to build my first Alexa skill to better understand the underlying <a href="https://developer.amazon.com/alexa-skills-kit">Alexa Skills Kit</a> that powers all Amazon Echo devices to give users this new intuitive voice interface. I was particularly interested in voice-user interface (VUI) design and how easy it would be to go from idea to building and then deploying a new skill. For the actual implementation, I decided to use the <a href="https://developer.amazon.com/blogs/post/Tx14R0IYYGH3SKT/flask-ask-a-new-python-framework-for-rapid-alexa-skills-kit-development">new Flask-Ask framework</a> by <a href="https://twitter.com/_johnwheeler">John Wheeler</a> to build the skill along with using <a href="https://developer.amazon.com/blogs/post/8e8ad73a-99e9-4c0f-a7b3-60f92287b0bf/new-alexa-tutorial-deploy-flask-ask-skills-to-aws-lambda-with-zappa">Zappa to deploy my Python code into AWS Lambda</a>. </div>
<div class="separator" style="clear: both; text-align: justify;">
<br /></div>
<div class="separator" style="clear: both; text-align: justify;">
I settled on a simple idea for a "job finder" skill that allows users to answer 2 simple questions that Alexa asks them about which city and job role they were interested in. The skill would then return the most relevant job listings that had been posted in the last 24 hours according to the answers given. I imagined users waking up in the morning with their coffee in hand and asking their Alexa "job finder" skill to get the latest updates for their ongoing job search. If the skill responded with some particularly timely and relevant jobs, the user could then decide to immediately open their laptop and apply for those jobs or, if not, the user could simply continue to enjoy their coffee and resume the job search later on in the day.</div>
<div class="separator" style="clear: both; text-align: justify;">
<br /></div>
<h2 style="clear: both; text-align: justify;">
Implementation</h2>
<div class="separator" style="clear: both; text-align: justify;">
<br /></div>
<div class="separator" style="clear: both; text-align: justify;">
I spent a few evenings coding up the "job finder" skill's Python backend and configuring the interaction model (the intent schema and sample utterances) to get it working locally. I ended up with a single job_finder.py file for the bulk of my application logic which is only 116 lines of code and a template.yaml file that contains a number of the templated launch, answer and re-prompt responses which are required for the skill. All-in-all the learning curve was reasonable and most of the learning came from understanding the <a href="https://developer.amazon.com/public/solutions/alexa/alexa-skills-kit/docs/alexa-skills-kit-interaction-model-reference">new type of interaction model</a> set out by the Alexa Skills Kit.</div>
<div style="text-align: justify;">
<br />
<h2>
Demo</h2>
<br />
As of writing this post the "job finder" skill is being reviewed by Amazon's Alexa's team and therefore is unavailable at present to enable on your own Amazon Echo device (the skill should be certified in a week or less). In the meantime I've recorded a demo using <a href="https://echosim.io/">Amazon Alexa Testing Tool called Echosim.io</a>:<br />
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<iframe frameborder="no" height="166" scrolling="no" src="https://w.soundcloud.com/player/?url=https%3A//api.soundcloud.com/tracks/332792030%3Fsecret_token%3Ds-96J4D&color=ff5500&auto_play=false&hide_related=false&show_comments=true&show_user=true&show_reposts=false" width="100%"></iframe></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<h2 style="clear: both; text-align: justify;">
Learnings</h2>
<div>
<br /></div>
<div>
<ul style="text-align: left;">
<li style="text-align: justify;">I've capped the number of jobs Alexa responds with to a maximum of 5. I think this is a manageable amount but given that Alexa might misunderstand the job role given by the user, it could be frustrating to have to listen to all 5 jobs if they are based on an incorrect interpretation. Therefore I would potentially improve the skill's VUI to either have Alexa repeat the job role specified by the user before all 5 jobs are listed out OR I would pause Alexa after the first job listed and ask the user a yes/no question on whether Alexa should continue or not.</li>
</ul>
<ul style="text-align: left;">
<li style="text-align: justify;">I've purposefully kept each job listing very brief so it's easy for users to consume while listening but it might be useful for users to get more detailed information. I could improve the skill's VUI by asking a follow-up question at the end of listing the 5 jobs to know whether or not the user wanted more details about any or all of the jobs listed.</li>
</ul>
<ul style="text-align: left;">
<li style="text-align: justify;">I'm using the AMAZON.ProfessionalType intent slot and this is currently only supported in the US. Therefore when the skill is certified by the Amazon team it will only be available on US based Alexa devices. I've asked about <a href="https://forums.developer.amazon.com/questions/72332/when-can-we-expect-the-amazonprofessionaltype-inte.html">UK support for the AMAZON.ProfessionalType intent slots</a> but have not yet received information about when UK support for it will be available. </li>
</ul>
<ul style="text-align: left;">
<li style="text-align: justify;">Analytics are very important for knowing what is and is not working from a user's perspective. I often use tools like Google Analytics for websites and Mixpanel/Amplitude for mobile apps which have been very insightful to better understand user behaviour. For my Alexa skill, I'd like to add in something like <a href="https://www.dashbot.io/">Dashbot</a> analytics to get a window into what users are and are not doing with the skill.</li>
</ul>
<ul style="text-align: left;">
<li style="text-align: justify;">Most importantly I would like to find ways to improve the VUI for subsequent invocations of the "job finder" skill. If a user has already performed a prior job search then I'd like to be able to have a search saved against their user profile and then have the skill look it up and ask the user whether or not it should be re-used. This will reduce the number of questions Alexa asks the user upon each new "job finder" skill invocation. Such functionality can be accomplished by identifying users by their <a href="http://flask-ask.readthedocs.io/en/latest/requests.html#session-context-request-and-version-context-locals">UserId obtained from the session</a> and then querying a database for prior jobs searches to re-use.</li>
</ul>
<div style="text-align: justify;">
<br />
<hr />
</div>
</div>
<div style="text-align: justify;">
<h2 style="clear: both;">
</h2>
<h2 style="clear: both;">
</h2>
<h2 style="clear: both;">
</h2>
<h2 style="clear: both;">
<br /></h2>
<h2 style="clear: both;">
Update</h2>
</div>
<div style="text-align: justify;">
<br />
<ol>
<li>I mentioned implementing analytics within the skill previously. I've since integrated <a href="http://voicelabs.co/">VoiceInsights analytics from VoiceLabs.co</a> to begin receiving data for individual and aggregate user sessions across metrics such as usage, pathing, speech and retention. I'm finding it more valuable compared to Alexa's skill metrics data that are provided by default.</li>
<li>My "Job Finder" skill is now live in both the <a href="https://www.amazon.com/Andrew-Thompson-Job-Finder/dp/B073VBNFCW/ref=sr_1_1">US</a> and <a href="https://www.amazon.co.uk/Andrew-Thompson-Job-Finder/dp/B073VBNFCW/ref=sr_1_1">UK</a> markets:</li>
</ol>
<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://2.bp.blogspot.com/-as410G_VkkQ/WXZtAiGr22I/AAAAAAAAKtU/KzRrN_7_I5IfYR4alOtZUFPI8txeyiVWQCLcBGAs/s1600/Screen%2BShot%2B2017-07-24%2Bat%2B22.55.48.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="880" data-original-width="1600" height="220" src="https://2.bp.blogspot.com/-as410G_VkkQ/WXZtAiGr22I/AAAAAAAAKtU/KzRrN_7_I5IfYR4alOtZUFPI8txeyiVWQCLcBGAs/s400/Screen%2BShot%2B2017-07-24%2Bat%2B22.55.48.png" width="400" /></a></div>
<br /></div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
</div>
</div>
Andrewhttp://www.blogger.com/profile/03252749023463514361noreply@blogger.com0tag:blogger.com,1999:blog-1847717784907730054.post-31610814366837190642017-06-29T18:21:00.003+01:002020-02-03T23:58:14.179+00:00My podcast with AWS Startup Stories about hiring for startups<div dir="ltr" style="text-align: left;" trbidi="on">
<div style="text-align: justify;">
On June 23rd, 2017 AWS released the final podcast in a 6 episode series titled <a href="https://aws.amazon.com/campaigns/aws-startups-stories">AWS Startup Stories</a>. The series of podcasts cover a range of topics from startup fundraising (<a href="https://www.crowdcube.com/">Crowdcube</a>, <a href="https://localglobe.vc/">LocalGlobe</a>) to scaling a startup (<a href="https://monzo.com/">Monzo</a>, <a href="https://www.starlingbank.com/">Starling Bank</a>, <a href="https://getonce.com/">Once</a> dating app) and growing and retaining a team (<a href="http://www.makersacademy.com/">Makers Academy</a>, <a href="http://yoyowallet.com/">Yoyo Wallet</a>).</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
In episode 6 <a href="https://twitter.com/isoworg">Will Bentinck</a>, Head of Careers at Makers Academy and <a href="https://twitter.com/andrewthompsonx">myself</a>, VP Engineering at Yoyo Wallet discuss the ins and outs of hiring for startups and how to retain talent long-term. The 30 minute podcast was hosted by <a href="https://twitter.com/darren_mowry">Darren Mowry</a>, Head of Startup Business Development and Venture Capital at AWS EMEA.</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
<iframe allow="autoplay" frameborder="no" height="166" scrolling="no" src="https://w.soundcloud.com/player/?url=https%3A//api.soundcloud.com/tracks/754641637&color=%23ff5500&auto_play=false&hide_related=false&show_comments=true&show_user=true&show_reposts=false&show_teaser=true" width="100%"></iframe>
</div>
</div>
Andrewhttp://www.blogger.com/profile/03252749023463514361noreply@blogger.com0tag:blogger.com,1999:blog-1847717784907730054.post-21429676202673934362017-05-01T21:57:00.000+01:002017-07-12T02:03:43.355+01:00Yoyo Wallet Powers The New Caffè Nero App<div dir="ltr" style="text-align: left;" trbidi="on">
<div style="text-align: justify;">
<h2>
Summary</h2>
<div>
<br />
<ol>
<li>We built the Caffè Nero App on top of the existing Yoyo platform</li>
<li>We achieved #1 in the Food & Beverage category on launch day ahead of Starbucks and UberEats</li>
<li>We have the highest rated app in its category (mostly 5 star reviews)</li>
<li>We modified our architecture and built a fully automated CI/CD system for rapid, regular releases</li>
</ol>
<div>
<br /></div>
</div>
<h2>
Launch</h2>
<br />
On April 10th, 2017 my team and I at Yoyo Wallet <a href="http://mobilemarketingmagazine.com/caffe-nero-loyalty-payment-app-marcus-denison-smith">launched the Caffè Nero App</a> across the UK and Ireland enabling a digital payment and loyalty experience for millions of Caffè Nero's customers.</div>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://1.bp.blogspot.com/-t86vQQUEr5I/WQHPrdRmSTI/AAAAAAAAFGw/9ksbTashfmotTxaaf3HELrwkpjTYOOQnwCLcB/s1600/nero_banner_v5.png" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" height="523" src="https://1.bp.blogspot.com/-t86vQQUEr5I/WQHPrdRmSTI/AAAAAAAAFGw/9ksbTashfmotTxaaf3HELrwkpjTYOOQnwCLcB/s640/nero_banner_v5.png" width="640" /></a></div>
<div class="separator" style="clear: both; text-align: justify;">
The product engineering team at Yoyo Wallet (consisting of Android, iOS and backend engineers along with product mangers and designers) were responsible for designing, building and shipping the Caffè Nero App for the April deadline, whilst still continuing to improve and maintain the Yoyo Wallet App which is live across the UK and Europe at many Universities (e.g. Imperial, Oxford Brookes, York), Corporates (e.g. JP Morgan, Guardian, Accenture) and high street stores (e.g. Planet Organic, independent coffee shops).</div>
<div class="separator" style="clear: both; text-align: justify;">
<br /></div>
<h2 style="text-align: justify;">
Marketing</h2>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://3.bp.blogspot.com/-p4rjBsOWl3g/WQJUvN30X_I/AAAAAAAAFHk/jhIkauAjRMM0eeTYSDqklp3W77jGL5WqwCLcB/s1600/in-store-collateral.jpeg" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><br /></a><a href="https://3.bp.blogspot.com/-p4rjBsOWl3g/WQJUvN30X_I/AAAAAAAAFHk/jhIkauAjRMM0eeTYSDqklp3W77jGL5WqwCLcB/s1600/in-store-collateral.jpeg" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" height="360" src="https://3.bp.blogspot.com/-p4rjBsOWl3g/WQJUvN30X_I/AAAAAAAAFHk/jhIkauAjRMM0eeTYSDqklp3W77jGL5WqwCLcB/s640/in-store-collateral.jpeg" width="640" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div class="separator" style="clear: both; text-align: justify;">
Nero kicked off a number of marketing campaigns after the release of the app to raise its profile, including posters and collateral in-store (see above) and email campaigns. </div>
<div class="separator" style="clear: both; text-align: justify;">
<br /></div>
<h2 style="text-align: justify;">
Feedback</h2>
<div class="separator" style="clear: both; text-align: justify;">
<br /></div>
<div class="separator" style="clear: both; text-align: justify;">
On launch day and the days since we've seen a surge in registrations for the app which resulted in the app obtaining the top ranked spot in the Apple App Store for the Free Food & Drink category (out ranking other apps such as Starbucks, Costa, UberEats, Nando's and Just Eat to name a few). Off the back of customers downloading, registering and using with the app to pay and collect stamps, we've received mostly 5 star reviews making us the top reviewed app in the category (As of April 27th we've received a total of 395 iOS and 416 Android reviews with ratings of 5.0 and 4.7 respectively). Along with positive user rankings, we've also maintained a very high quality threshold shown by our crash-free session rates which are at 99.97% and 99.94% for the latest versions of the iOS and Android apps respectively (measured via Fabric's SDK).</div>
<div class="separator" style="clear: both; text-align: justify;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://2.bp.blogspot.com/-zRL5XADOZL8/WQHbsNHdDGI/AAAAAAAAFHU/b2JpIgX6cYcwctOOZBILp7mXU_jJNWZMQCLcB/s1600/app-store-ranking-reviews.jpeg" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" height="360" src="https://2.bp.blogspot.com/-zRL5XADOZL8/WQHbsNHdDGI/AAAAAAAAFHU/b2JpIgX6cYcwctOOZBILp7mXU_jJNWZMQCLcB/s640/app-store-ranking-reviews.jpeg" width="640" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div style="text-align: justify;">
<h2>
Features</h2>
<br />
We've abstracted away much of the complexity of the app behind a simple UI/UX that customers engage with and rate highly. Behind this simple UI/UX the app has the following features:<br />
<br /></div>
<ul style="text-align: left;">
<li><b style="text-align: justify;">Multi-mode registration</b><span style="text-align: justify;"> (Email, Facebook or Yoyo login)</span></li>
<li style="text-align: justify;">Camera based <b>credit card scanning</b> to more easily gather card details (A/B testing proved the uplift in conversion)</li>
<li style="text-align: justify;"><b>Secure payment and loyalty</b> in a single scan of the app OR <b>loyalty-only</b> so customers can also pay with their own cash/card</li>
<li style="text-align: justify;"><b>Rule-based loyalty</b> that triggers in real-time based on each items in a customer's basket and notifies them via silent push notifications of their loyalty reward(s)</li>
<li style="text-align: justify;">An <b>activity feed</b> that aggregates receipts and loyalty reward information </li>
<li style="text-align: justify;"><b>Store locator</b> for finding the closest store</li>
<li style="text-align: justify;">Apple and Android <b>Wallet integration</b> for loyalty-only transactions</li>
<li style="text-align: justify;"><b>Settings</b>: account management, support and promo codes</li>
</ul>
<div style="text-align: justify;">
<br />
<h2>
Architecture</h2>
<br /></div>
<div style="text-align: justify;">
Given that Yoyo is powering the Caffè Nero app, as an engineering team we were very conscious of leveraging both our existing mobile and backend platform architecture in the right way to minimise code/feature duplication across 2 (and eventually more) mobile frontends. Both sets of mobile apps call the same Yoyo API endpoints running on Amazon Web Services (AWS) and much of the mobile code base, for both iOS and Android, is re-used inside our own internal SDK as well as the core business logic of the app which we call "App Core". Only the UI of the mobile apps (which is visually different) and a small bit of custom business logic are unique to each app and therefore cannot be re-used:</div>
<div>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://4.bp.blogspot.com/-lxq4Xgv81GI/WQHLLb-okEI/AAAAAAAAFGg/FwHsJDrZwGAEI_EbKSFAfyCl1bTvAyyngCLcB/s1600/multi-apps.001.jpeg" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" height="360" src="https://4.bp.blogspot.com/-lxq4Xgv81GI/WQHLLb-okEI/AAAAAAAAFGg/FwHsJDrZwGAEI_EbKSFAfyCl1bTvAyyngCLcB/s640/multi-apps.001.jpeg" width="640" /></a></div>
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<div style="text-align: justify;">
<h2>
Shipping</h2>
<br />
At Yoyo we ship new versions of <b>ALL</b> our mobile apps to their respective app stores every week without fail on Tuesdays at 3pm for iOS and Tuesdays and Thursdays at 3pm for Android (we follow a <a href="https://en.wikipedia.org/wiki/Software_release_train">release train process</a>). This requires building, packaging and deploying a total of 6 .ipa and .apk binaries across our internal test, staging and production environments along with our 3 testing channels (alpha, beta and live). To manage this complexity effectively we've automated the entire build, package and deployment process for multiple apps by using our own custom mobile CI/CD pipeline built on <a href="https://www.bitrise.io/">Bitrise</a>, <a href="https://get.fabric.io/">Fabric</a>, <a href="https://developer.apple.com/testflight/">Testflight</a> and our own scripts. </div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
Here is an example of what our mobile CI/CD pipeline does each day multiple times:<br />
<ol>
<li>An iOS engineer fixes a bug in the SDK and merges his/her change to master</li>
<li>Bitrise immediately kicks off a new CI build cycle for both the Yoyo Wallet and Caffè Nero versions of the iOS app</li>
<li>Once complete both iOS .ipa binaries are pushed out via Fabric to all alpha testers</li>
<li>The next time all alpha testers open their Yoyo Wallet or Caffè Nero app on their iPhones (most internal Yoyo staff are alpha testers except the sales team) they will be asked to download the latest build</li>
<li>Once downloaded, alpha testers will be using the latest and greatest version of the app and are more easily able to identify bugs or usability issues</li>
</ol>
<div>
This new, fully automated, CI/CD pipeline was implemented late in 2016 and has dramatically increased our mobile team's ability to rapidly release new versions of the mobile app to our users. The chart below shows this increase over time as we've improved our ability to ship to production using automation:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://4.bp.blogspot.com/-J4D8Rc2hepI/WQhOA2Wh7RI/AAAAAAAAFII/tN4V8O2VX7Aplroanug383iqfvaSJovhQCLcB/s1600/yearly-app-releases.jpeg" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" height="360" src="https://4.bp.blogspot.com/-J4D8Rc2hepI/WQhOA2Wh7RI/AAAAAAAAFII/tN4V8O2VX7Aplroanug383iqfvaSJovhQCLcB/s640/yearly-app-releases.jpeg" width="640" /></a></div>
<br /></div>
<div>
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div>
It's worth mentioning that we also use a company called <a href="https://www.applause.com/">Applause</a> for crowdsourced continuous testing of our mobile apps. The service that Applause provides is incredibly helpful and cost-effective in terms of finding many usability issues and bugs across the <a href="https://www.quora.com/What-is-Android-fragmentation-exactly">plethora of Android devices and configurations</a> that are available in the marketplace today. However we <b>do not</b> use their crowdsourced continuous testing service as a quality assurance gatekeeper for pending production app deployments (there is a very specific reason for why we do not use them in this way which I will explain further). </div>
<div>
<br /></div>
<div>
Whenever there is a human gatekeeper of quality, I find that developers and their teams often rely on that person or set of people as the main quality assurance mechanism instead of themselves (they are less thorough with their code changes and code reviews and often write fewer automated tests). They effectively offload the burden of responsibility for quality assurance to someone else for the change(s) they are making. This is orthogonal to the idea that each developer is the best person to know what could be most affected by their changes and should therefore assure the quality themselves. At Yoyo we wanted to keep quality high by using Applause but did not want the service to become a crutch for developers to rely on. Therefore we only use Applause in parallel with existing deployments to production. Applause testing never stops any deployment, it merely identifies existing issues and bugs that need to be fixed post deployment and does a very good job at this task.</div>
<div>
<br /></div>
<div>
<h2>
Learnings</h2>
</div>
<div>
<br /></div>
<div>
<ul>
<li><b>Watch out for trademark checks</b>: One unexpected thing to happen in the 11th hour was when we submitted the Android app to Google Play and it was rejected. We had completed and submitted the iOS app a week earlier because we knew the review process was more stringent and always took longer with Apple. We weren't worrying about our submission to Google Play as it was always seamless due to less checks and balances (or so we thought). This was not what happened in our case, Google flagged the fact that a 3rd party developer (i.e. us as in Yoyo Wallet) were submitting an app with someone else's trademarked branding and so the app was rejected. We had to scramble in the 11th hour over a weekend to get documents signed by Caffè Nero to assure Google that we, as a 3rd party developer, had the right to submit an app that contained Caffè Nero branding. This did delay our Android submission by a few days but luckily we still had enough head room for it to ship on-time.</li>
</ul>
<ul>
<li><b>Public wifi with logins can cause headaches</b>: A phone's OS will automatically connect to a public wifi that it has previously connected to when it's within range but the user usually still has to accept the terms and/or re-login to gain full internet connectivity. If the latter does not occur then the user usually assumes they have internet connectivity when the app is open due to the wifi symbol displaying on their phone when in fact they do not have internet connectivity. There are a few ways to solve this by either (1) if the public wifi is within your control make sure certain outbound connections (via IP or domain) are given access even without the need to accept the terms or re-login to the public wifi OR (2) the app displays a "no internet connectivity" warning message so users expect limited functionality until they have fully connected to the internet. </li>
</ul>
<ul>
<li><b>Improving mobile architecture and deployment infrastructure upfront was worth it</b>: Given that we had the Yoyo Wallet app and were effectively white labelling it (modifying the UI for Caffè Nero as well as adding some new features), we wanted to approach this intelligently to minimise long-term technical debt. We knew that white labelling Yoyo Wallet could get rather messy quite quickly with multiple sets of apps being created and maintained long-term. Yoyo's mobile team spent time upfront to improve the mobile architecture on both iOS and Android to separate out the App SDK and App Core dependency layers which could be re-used across apps while the UI layer could be modified heavily. We also focused on improving the deployment infrastructure using Bitrise CI and compile time configuration flags within the app so that we could automatically build the different UI versions of the app with the reusable App SDK and App Core dependencies. Although this deployment infrastructure took time upfront to build and configure correctly, it reaped large dividends later in the project as the deadline loomed (it continues to reap ongoing benefits every day with each new commit). Mobile engineers now barely need to think about the Yoyo Wallet or Caffè Nero app deployments, we focus on improving the UI or business logic of the app and the deployments continue to happen automatically each day and in exactly the same way since the start of the Caffè Nero app project.</li>
</ul>
</div>
<div>
<br />
<h2>
Conclusion</h2>
<br /></div>
</div>
<div style="border: 0px; line-height: 1.42857em; margin: 0px; padding: 0px;">
<div style="border: 0px; line-height: 1.42857em; margin: 0px; padding: 0px;">
<div style="border: 0px; line-height: 1.42857em; margin: 0px; padding: 0px;">
<div style="text-align: justify;">
The entire team at Yoyo Wallet have done an incredible job building Caffè Nero's mobile app on time for the April 10th launch. Not only did they meet the deadline but have built an app that is available to millions of Caffè Nero customers. Many of these customers already love the product based on their ratings and are actively asking for more functionality to be added in near the future which is exactly the kind of response a product engineering team hopes for when releasing a major new product into the market.</div>
<div style="text-align: justify;">
<br /></div>
<hr style="line-height: 1.42857em; text-align: justify;" />
</div>
</div>
</div>
<br />
Download the Caffè Nero App today:<br />
<ul style="text-align: left;">
<li>Apple App Store: <a href="https://itunes.apple.com/gb/app/caff%C3%A8-nero/id1209864393?mt=8">https://itunes.apple.com/gb/app/caff%C3%A8-nero/id1209864393?mt=8</a></li>
<li>Google Play: <a href="https://play.google.com/store/apps/details?id=com.yoyowallet.caffenero&hl=en_GB">https://play.google.com/store/apps/details?id=com.yoyowallet.caffenero&hl=en_GB</a></li>
</ul>
<div>
<br /></div>
</div>
<div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://2.bp.blogspot.com/-13idXXJRvw4/WNr0whoJWEI/AAAAAAAAFFo/uYBFcNl5Q6cP9ZDodKqA_kpTvGUfFEpWgCLcB/s1600/Caffe%2BNero%2BFeatures%2BiOS.png" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-left: 1em; text-align: left;"><img border="0" height="284" src="https://2.bp.blogspot.com/-13idXXJRvw4/WNr0whoJWEI/AAAAAAAAFFo/uYBFcNl5Q6cP9ZDodKqA_kpTvGUfFEpWgCLcB/s640/Caffe%2BNero%2BFeatures%2BiOS.png" width="640" /></a></div>
</div>
</div>
Andrewhttp://www.blogger.com/profile/03252749023463514361noreply@blogger.com0London, UK51.5073509 -0.1277582999999822351.1912379 -0.77320529999998222 51.8234639 0.51768870000001777tag:blogger.com,1999:blog-1847717784907730054.post-22830698455871534662015-07-12T23:57:00.000+01:002015-08-27T23:58:21.895+01:0010 Steps To Priming Kanban<div dir="ltr" style="text-align: left;" trbidi="on">
<div style="text-align: justify;">
<a href="https://twitter.com/j_boeg">Jesper Boeg</a> has written one of the best and most succinct PDF docs on Kanban (43 pages in all). He breaks Kanban down into 10 steps that can be implemented piecemeal until your team has a comprehensive Kanban process governing all aspects of software development.</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: center;">
<iframe allowfullscreen="" frameborder="0" height="510" marginheight="0" marginwidth="0" scrolling="no" src="//www.slideshare.net/slideshow/embed_code/key/oJyD0IWOE9J9q" style="border-width: 1px; border: 1px solid #CCC; margin-bottom: 5px; max-width: 100%;" width="477"> </iframe> </div>
<div style="margin-bottom: 5px;">
<div style="text-align: center;">
<span style="font-size: x-small;"><strong> <a href="https://www.slideshare.net/andrewmatthewthompson/priming-kanban-by-jesper-boeg" target="_blank" title="Priming Kanban by Jesper Boeg">Priming Kanban by Jesper Boeg</a> </strong> from <strong><a href="https://www.slideshare.net/andrewmatthewthompson" target="_blank">Andrew Thompson</a></strong></span></div>
</div>
</div>
Andrewhttp://www.blogger.com/profile/03252749023463514361noreply@blogger.com0tag:blogger.com,1999:blog-1847717784907730054.post-16385488603136352702015-05-28T22:41:00.000+01:002015-08-27T22:50:45.402+01:00Solarwinds Press Release for Pingdom, Librato and Papertrail<div dir="ltr" style="text-align: left;" trbidi="on">
<div style="text-align: justify;">
<a href="http://www.solarwinds.com/">Solarwinds</a>, the Texas based enterprise IT management company that purchased <a href="http://royal.pingdom.com/2014/06/18/pingdom-joining-solarwinds/">Pingdom for an undisclosed amount</a>, <a href="http://blog.librato.com/posts/librato-solarwinds-press-release">Librato for $40M</a> and <a href="http://techcrunch.com/2015/04/28/solarwinds-acquires-log-management-service-papertrail-for-41m-in-cash/">Papertrail for $41M</a> over the last couple years, recently got in touch with me because my company, <a href="http://www.insignum.io/">Insignum</a>, uses all three of their products to manage our machine learning infrastructure in the cloud.</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
Solarwinds were attending Velocity 2015 in Santa Clara and asked for my take on their products to include in a <a href="http://www.solarwinds.com/company/newsroom/press_releases/sw_cloud_to_showcase_full_stack_cloud_monitoring_at_velocity_santa_clara_2015.aspx">press release</a>. Here is the excerpt that includes my comments:</div>
<blockquote class="tr_bq" style="text-align: justify;">
“All of these SolarWinds Cloud products work well together as they're solving slightly different problems at each level of our technology stack,” said Andrew Thompson, co-founder and CEO of Insignum, an automated analytics intelligence company. “Pingdom is our go-to solution for understanding the overall health and stability of the web pages our customers engage with day-to-day, while Librato gives us an understanding of how all our infrastructure is performing to deliver value to users, and then when something breaks, Papertrail becomes the service that we use to drill into the exact details of the problem so we can devise a solution – its true value is simplicity.”</blockquote>
</div>
Andrewhttp://www.blogger.com/profile/03252749023463514361noreply@blogger.com0tag:blogger.com,1999:blog-1847717784907730054.post-50097399624574924942014-09-17T23:19:00.000+01:002015-08-28T20:52:05.097+01:00Improving On-Site Search Functionality For GrowthHackers.com<div dir="ltr" style="text-align: left;" trbidi="on">
<div style="text-align: justify;">
Back in 2013 when <a href="http://growthhackers.com/">GrowthHackers.com</a> started gaining prominence and was regularly used by thousands of growth engineers, I started noticing that their on-site search functionality needed some improvement as relevant content was hard to find. I knew the small GrowthHackers.com team were working hard to design and deliver important features that were growing the community, so I wanted to help out by articulating the problem and devising a solution that would be easy for them to implement and could almost immediately be leveraged by the GrowthHackers.com community. </div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
Here is the short presentation I created and sent to <a href="https://twitter.com/SeanEllis">Sean Ellis</a>. It was subsequently used to improve their on-site search and is still used today by the GrowthHackers.com community (note that instead of using <a href="https://swiftype.com/">Swiftype</a> they went with a similar product called <a href="https://www.algolia.com/">Algolia</a>):</div>
<div style="text-align: justify;">
<div style="text-align: center;">
<br /></div>
</div>
<div style="text-align: center;">
<iframe allowfullscreen="" frameborder="0" height="355" marginheight="0" marginwidth="0" scrolling="no" src="//www.slideshare.net/slideshow/embed_code/key/tIPEO3SKrDQjK" style="border-width: 1px; border: 1px solid #CCC; margin-bottom: 5px; max-width: 100%;" width="425"> </iframe> </div>
<div style="margin-bottom: 5px;">
<div style="text-align: center;">
<span style="font-size: x-small;"><strong> <a href="https://www.slideshare.net/andrewmatthewthompson/improving-search-on-growthhackers" target="_blank" title="Improving Search on GrowthHackers.com">Improving Search on GrowthHackers.com</a> </strong> from <strong><a href="https://www.slideshare.net/andrewmatthewthompson" target="_blank">Andrew Thompson</a></strong></span> <br />
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
The Swiftype based search for GrowthHackers.com that I setup when I created the above presentation is still working today (give it a try):<br />
<form>
<div style="text-align: center;">
<input class="st-search-input" id="st-search-input" type="text" /></div>
</form>
<script type="text/javascript">
var Swiftype = window.Swiftype || {};
(function() {
Swiftype.key = 'X9kUJi9vNpsZTq3i1pjg';
/** DO NOT EDIT BELOW THIS LINE **/
var script = document.createElement('script'); script.type = 'text/javascript'; script.async = true;
script.src = "//swiftype.com/embed.js";
var entry = document.getElementsByTagName('script')[0];
document.getElementsByTagName('script')[0].parentNode.insertBefore(script, entry);
}());
</script>
</div>
</div>
</div>
</div>
Andrewhttp://www.blogger.com/profile/03252749023463514361noreply@blogger.com0tag:blogger.com,1999:blog-1847717784907730054.post-6434038372381543152014-07-20T10:26:00.000+01:002014-07-20T10:29:43.507+01:00Just launched my new company called Insignum<div dir="ltr" style="text-align: left;" trbidi="on">
<div style="text-align: justify;">
Yesterday I launched my new company called <a href="http://www.insignum.io/">Insignum</a>. It delivers anomaly detection and notification for your analytics across Mixpanel, KISSmetrics and Google Analytics.</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
The product provides data-driven startups with automated analytics intelligence to find deviations in expected customer behaviour as they happen. Using machine learning algorithms, it continuously searches your data and when an anomaly is detected it will notify you so that you can act on the insight.</div>
<div>
<br /></div>
<div>
<a href="http://www.insignum.io" target="_blank" imageanchor="1" style="clear: left; color: #231f20; font-family: 'Varela Round', sans-serif; font-size: 44px; line-height: 1.5em; margin-bottom: 1em; margin-right: 1em;"><img border="0" src="http://1.bp.blogspot.com/-hJr5m2IrvZA/U8uH4GjLfNI/AAAAAAAAAvE/gnodgl8iNLQ/s1600/Insignum+Homepage.png" height="401" width="640" /></a></div>
<div>
<br /></div>
<div>
If you have any feedback about the product I'd be very interested in hearing about it. Insignum's Twitter account is <a href="https://twitter.com/insignum_io">@insignum_io</a>.</div>
<div>
<div style="text-align: right;">
</div>
</div>
</div>
Andrewhttp://www.blogger.com/profile/03252749023463514361noreply@blogger.com0tag:blogger.com,1999:blog-1847717784907730054.post-39008215663726501602014-06-03T16:32:00.000+01:002014-07-20T10:20:07.307+01:00How a one word change increased product demo conversions by 139%<div dir="ltr" style="text-align: left;" trbidi="on">
<div style="box-sizing: border-box; color: #585858; line-height: 23px; text-align: justify;">
<span style="font-family: Times, Times New Roman, serif;">The following is a repost from the <a href="https://gocardless.com/blog/one-word-abtest/">GoCardless Blog</a> where I explained one of our successful A/B tests. I also started a good discussion about the merits of the A/B test over at <a href="http://growthhackers.com/how-a-one-word-change-increased-product-demo-conversions-by-139">growthhackers.com</a>.</span></div>
<div style="box-sizing: border-box; color: #585858; line-height: 23px; text-align: justify;">
<span style="font-family: Times, Times New Roman, serif;"><br /></span></div>
<div style="border: 0px; line-height: 1.428571em; margin: 0px; padding: 0px;">
<div style="border: 0px; line-height: 1.428571em; margin: 0px; padding: 0px;">
<div style="border: 0px; line-height: 1.428571em; margin: 0px; padding: 0px;">
<hr style="line-height: 1.428571em; text-align: justify;" />
</div>
</div>
</div>
<div style="border: 0px; line-height: 1.428571em; margin: 0px; padding: 0px; text-align: justify;">
<span style="font-family: Times, Times New Roman, serif;"><br /></span></div>
<div style="box-sizing: border-box; color: #585858; line-height: 23px; text-align: justify;">
<span style="font-family: Times, Times New Roman, serif;">This post looks at an A/B test with a simple copy change and how it improved conversion rates by 139%. The idea behind the A/B test was to give users immediate access to the product, via a recorded demo, instead of having them receive a personal phone call from our sales team later on.</span></div>
<h2 style="box-sizing: border-box; color: #2f2f2f; line-height: 23px; margin-top: 28px; text-align: justify;">
<span style="font-family: Times, Times New Roman, serif;">
Theory</span></h2>
<div style="box-sizing: border-box; color: #585858; line-height: 23px; text-align: justify;">
<span style="font-family: Times, Times New Roman, serif;">Using a framework, when A/B testing potential improvements to landing pages, is helpful. Sean Ellis has a simple one for understanding the broad levers that can been affected: <a href="https://twitter.com/SeanEllis/status/231395889627938816" style="box-sizing: border-box; color: #3288ca;">Conversion Rate = Desire - Friction</a>. Desire and Friction can be further broken out and the <a href="http://www.widerfunnel.com/solutions/our-process/lift-model" style="box-sizing: border-box; color: #3288ca;">LIFT Model by WiderFunnel</a> describes this well:</span></div>
<div style="box-sizing: border-box; color: #585858; line-height: 23px;">
<span style="font-family: Times, Times New Roman, serif;"><br /></span></div>
<div style="box-sizing: border-box; color: #585858; line-height: 23px; text-align: center;">
<span style="font-family: Times, Times New Roman, serif;"><img alt="LIFT Framework for CRO" src="https://i.imgur.com/9EOOI0q.png" style="border: 0px; box-sizing: border-box; height: auto; max-width: 100%; vertical-align: middle;" /></span></div>
<div style="box-sizing: border-box; color: #585858; line-height: 23px;">
<span style="font-family: Times, Times New Roman, serif;"><br /></span></div>
<h2 style="box-sizing: border-box; color: #2f2f2f; line-height: 23px; margin-top: 28px; text-align: justify;">
<span style="font-family: Times, Times New Roman, serif;">
Hypothesis</span></h2>
<div style="box-sizing: border-box; color: #585858; line-height: 23px; text-align: justify;">
<span style="font-family: Times, Times New Roman, serif;">Our goal was to improve the conversion rate for demo requests so that customers could access the content they were interested in as soon as possible and with minimal friction. We wondered if the “Request a demo” button might be causing some users anxiety (as described by the LIFT model) and, as such, artificially lowering conversion rates. We then tested whether the wording “Watch a demo” would outperform the original “Request a demo” wording.</span></div>
<div style="box-sizing: border-box; color: #585858; line-height: 23px; text-align: justify;">
<span style="font-family: Times, Times New Roman, serif;">We had further reason to believe that immediate access to a recorded demo would be beneficial as only 1/5th of leads ended up watching the live demo which has been scheduled by our sales team.</span></div>
<h2 style="box-sizing: border-box; color: #2f2f2f; line-height: 23px; margin-top: 28px; text-align: justify;">
<span style="font-family: Times, Times New Roman, serif;">
Website Modifications</span></h2>
<div style="box-sizing: border-box; color: #585858; line-height: 23px; text-align: justify;">
<span style="font-family: Times, Times New Roman, serif;">The original GoCardless user experience with the “Request a demo” was the following:</span></div>
<ul style="box-sizing: border-box; color: #585858; line-height: 23px; margin-left: 20px;">
<li style="box-sizing: border-box; margin-bottom: 15px; text-align: justify;"><span style="font-family: Times, Times New Roman, serif;">Call to action (CTA) on the homepage was “Request a demo”</span></li>
<li style="box-sizing: border-box; margin-bottom: 15px; text-align: justify;"><span style="font-family: Times, Times New Roman, serif;">Users were taken to a request a demo form to fill out and submit</span></li>
<li style="box-sizing: border-box; margin-bottom: 15px; text-align: justify;"><span style="font-family: Times, Times New Roman, serif;">Upon completing the form, users were given a date and time of an upcoming live demo that they could participate in.</span></li>
</ul>
<div style="text-align: justify;">
<span style="color: #585858;"><span style="font-family: Times, Times New Roman, serif; line-height: 23px;"><br /></span></span></div>
<div style="box-sizing: border-box; color: #585858; line-height: 23px; text-align: center;">
<span style="font-family: Times, Times New Roman, serif;"><img alt="GoCardless Homepage With Request Copy" naptha_cursor="text" src="https://i.imgur.com/sUzhS7x.png" style="border: 0px; box-sizing: border-box; height: auto; max-width: 100%; vertical-align: middle;" /></span></div>
<div style="box-sizing: border-box; color: #585858; line-height: 23px;">
<span style="font-family: Times, Times New Roman, serif;"><br /></span></div>
<div style="box-sizing: border-box; color: #585858; line-height: 23px; text-align: center;">
<span style="font-family: Times, Times New Roman, serif;"><img alt="GoCardless Demo Page With Request Copy" naptha_cursor="region" src="https://i.imgur.com/XKdLWtP.png" style="border: 0px; box-sizing: border-box; height: auto; max-width: 100%; vertical-align: middle;" /></span></div>
<div style="box-sizing: border-box; color: #585858; line-height: 23px;">
<span style="font-family: Times, Times New Roman, serif;"><br /></span></div>
<div style="box-sizing: border-box; color: #585858; line-height: 23px; text-align: justify;">
<span style="font-family: Times, Times New Roman, serif;">We altered the user experience by giving users immediate access to a recorded demo:</span></div>
<ul style="box-sizing: border-box; color: #585858; line-height: 23px; margin-left: 20px;">
<li style="box-sizing: border-box; margin-bottom: 15px; text-align: justify;"><span style="font-family: Times, Times New Roman, serif;">Call to action (CTA) on the homepage is “Watch a demo”</span></li>
<li style="box-sizing: border-box; margin-bottom: 15px; text-align: justify;"><span style="font-family: Times, Times New Roman, serif;">Users are taken to a “Watch a demo” form to fill out and submit</span></li>
<li style="box-sizing: border-box; margin-bottom: 15px; text-align: justify;"><span style="font-family: Times, Times New Roman, serif;">Upon completing the form, users are shown a 10 minute recorded demo in their browser</span></li>
</ul>
<div style="box-sizing: border-box; color: #585858; line-height: 23px; text-align: justify;">
<span style="font-family: Times, Times New Roman, serif;"><img alt="GoCardless Homepage With Watch Copy" src="https://i.imgur.com/R4CdzOl.png" style="border: 0px; box-sizing: border-box; height: auto; max-width: 100%; vertical-align: middle;" /></span></div>
<div style="box-sizing: border-box; color: #585858; line-height: 23px;">
<span style="font-family: Times, Times New Roman, serif;"><br /></span></div>
<div style="box-sizing: border-box; color: #585858; line-height: 23px; text-align: center;">
<span style="font-family: Times, Times New Roman, serif;"><img alt="GoCardless Requesting A Demo Page With Watch Copy" src="https://i.imgur.com/Fl4wa0C.png" style="border: 0px; box-sizing: border-box; height: auto; max-width: 100%; vertical-align: middle;" /></span></div>
<div style="box-sizing: border-box; color: #585858; line-height: 23px;">
<span style="font-family: Times, Times New Roman, serif;"><br /></span></div>
<div style="box-sizing: border-box; color: #585858; line-height: 23px; text-align: center;">
<span style="font-family: Times, Times New Roman, serif;"><img alt="GoCardless Watch A Demo Page" src="https://i.imgur.com/MrqAv5x.png" style="border: 0px; box-sizing: border-box; height: auto; max-width: 100%; vertical-align: middle;" /></span></div>
<div style="box-sizing: border-box; color: #585858; line-height: 23px;">
<span style="font-family: Times, Times New Roman, serif;"><br /></span></div>
<div style="box-sizing: border-box; color: #585858; line-height: 23px; text-align: justify;">
<span style="font-family: Times, Times New Roman, serif;">Most of these changes were implemented within <a href="http://blog.optimizely.com/2013/01/03/multi-page-experiments-funnel-ab-testing/" style="box-sizing: border-box; color: #3288ca;">Optimizely’s Multi-page Experiments feature</a> (aka Conversion Funnel Testing). However, we did build the recorded demo page ourselves as we didn’t need to A/B test this page directly.</span></div>
<div style="box-sizing: border-box; color: #585858; line-height: 23px; text-align: justify;">
<span style="font-family: Times, Times New Roman, serif;"><br /></span></div>
<div style="box-sizing: border-box; color: #585858; line-height: 23px; text-align: justify;">
<span style="font-family: Times, Times New Roman, serif;">Although some time and effort went into thinking about reducing friction, the work required to implement and instrument these changes was very easy because of Optimizely and Mixpanel. Optimizely has a very useful single toggle option for sending super properties to Mixpanel so that we can track what happens deep within our funnel.</span></div>
<h2 style="box-sizing: border-box; color: #2f2f2f; line-height: 23px; margin-top: 28px; text-align: justify;">
<span style="font-family: Times, Times New Roman, serif;">
Results</span></h2>
<div style="box-sizing: border-box; color: #585858; line-height: 23px; text-align: justify;">
<span style="font-family: Times, Times New Roman, serif;">Given our acquisition channel characteristics, we ran the A/B test for a full 7 days. We then looked for a statistically significant winner with at least a 95% confidence level. Optimizely’s report panel below shows that the “Watch” version consistently outperformed the original version:</span></div>
<div style="box-sizing: border-box; color: #585858; line-height: 23px;">
<span style="font-family: Times, Times New Roman, serif;"><br /></span></div>
<div style="box-sizing: border-box; color: #585858; line-height: 23px; text-align: center;">
<span style="font-family: Times, Times New Roman, serif;"><img alt="Results From Optimizely Report" src="https://i.imgur.com/IFeZktm.png" style="border: 0px; box-sizing: border-box; height: auto; max-width: 100%; vertical-align: middle;" /></span></div>
<div style="box-sizing: border-box; color: #585858; line-height: 23px;">
<span style="font-family: Times, Times New Roman, serif;"><br /></span></div>
<div style="box-sizing: border-box; color: #585858; line-height: 23px; text-align: justify;">
<span style="font-family: Times, Times New Roman, serif;">We also ran the numbers through <a href="https://mixpanel.com/labs/split-test-calculator" style="box-sizing: border-box; color: #3288ca;">Mixpanel’s split test calculator</a> based on event data we track in our conversion funnels:</span></div>
<div style="box-sizing: border-box; color: #585858; line-height: 23px;">
<span style="font-family: Times, Times New Roman, serif;"><br /></span></div>
<div style="box-sizing: border-box; color: #585858; line-height: 23px; text-align: center;">
<span style="font-family: Times, Times New Roman, serif;"><img alt="Results From Mixpanel's Split Test Calculator" naptha_cursor="text" src="https://i.imgur.com/GPpzwSy.png" style="border: 0px; box-sizing: border-box; height: auto; max-width: 100%; vertical-align: middle;" /></span></div>
<div style="box-sizing: border-box; color: #585858; line-height: 23px;">
<span style="font-family: Times, Times New Roman, serif;"><br /></span></div>
<div style="box-sizing: border-box; color: #585858; line-height: 23px; text-align: justify;">
<span style="font-family: Times, Times New Roman, serif;">This shows that the “Watch a demo” version is more than twice as effective as the “Request a demo” version (139% increase in conversion). With this simple copy change, derived from the idea of reducing friction for new users, we’ve dramatically increased the number of users who watch a product demo and are therefore more likely to become customers.</span></div>
</div>
Andrewhttp://www.blogger.com/profile/03252749023463514361noreply@blogger.com0tag:blogger.com,1999:blog-1847717784907730054.post-84622966752804323692013-12-18T05:40:00.000+00:002013-12-18T06:01:38.984+00:0010 Steps For Running A/B Tests On Mobile Apps<div dir="ltr" style="text-align: left;" trbidi="on">
<div class="page" title="Page 4">
<div class="section">
<div class="section">
<div class="layoutArea">
<div class="column">
<div class="separator" style="background-color: white; clear: both; text-align: center;">
<span style="color: black; font-family: inherit; margin-left: 1em; margin-right: 1em;"><a href="http://2.bp.blogspot.com/-XGR94_shPOY/UrDkAmCekLI/AAAAAAAAAkQ/8-SuHnK8e40/s1600/A:B+Test+-+Version+A+vs+Version+B.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="278" src="http://2.bp.blogspot.com/-XGR94_shPOY/UrDkAmCekLI/AAAAAAAAAkQ/8-SuHnK8e40/s400/A:B+Test+-+Version+A+vs+Version+B.png" width="400" /></a></span></div>
<div class="separator" style="background-color: white; clear: both; text-align: justify;">
<br /></div>
<div class="separator" style="background-color: white; clear: both; text-align: justify;">
<span style="background-color: transparent;">There are a number of mobile architectures that support effective A/B testing within mobile apps. They range from rapid prototyping ones based on HTML5 components to feature flag based ones that trigger different versions of native components. The trade-offs are between in-app performance, testing iteration time and the native look and feel within the app. The main concern for effective A/B testing is to produce as many valid experiments as possible in the shortest amount of time. Therefore the longer this process takes, the longer it will take to discover what version(s) of the app perform best for various user segments. Whichever strategy is used, A/B tests should not be dependent on infrequent App Store releases to be the most effective.</span></div>
<div class="separator" style="background-color: white; clear: both; text-align: justify;">
<span style="background-color: transparent;"><br /></span></div>
<div class="separator" style="background-color: white; clear: both; text-align: justify;">
<span style="background-color: transparent;">After setting up a new A/B testing framework, its important to run an A/A test and determine if it is calibrated correctly. This type of A/A test should also be run every so often to make sure the A/B testing framework still works as expected and produces the correct statistical results.</span></div>
<div class="separator" style="background-color: white; clear: both; text-align: justify;">
<span style="background-color: transparent;"><br /></span></div>
<div class="separator" style="clear: both;">
<span style="background-color: transparent;">Once a basic A/B testing framework is setup, here are the steps to run an effective A/B test:</span></div>
<ol style="text-align: left;">
<li><b><u>Define</u></b> a goal that can be accurately measured. The effort in this step will reap dividends later in reducing the number of failed or ineffective tests. </li>
<li><b><u>Brainstorm</u></b> ideas for how to satisfy the goal. These can come from a variety of places such as qualitative customer feedback, employee suggestions, behavioural economic theories, gut feelings about product improvements, etc. </li>
<li><b><u>Prioritize</u></b> the list of ideas above based on the ease of implementation, the estimation of improvement potential and the relative position in the funnel. </li>
<li><b><u>Setup</u></b> the necessary event-based analytics tracking for an individual user's flow through the entire app. These events should be wired together to produce a funnel so that it is clear what the conversion rates are at each step. Depending on what is being tested, the user’s flow should begin from their entry point in the app (direct launch, push notification or website launch) through to the point of purchase and/or post-purchase follow-up. Another important strategy is to measure not only the success of the step being tested, but also the overall engagement of a user. </li>
<li><b><u>Capture</u></b> a baseline set of metrics for how the app currently performs for various user segments before any testing is run. </li>
<li><b><u>Build</u></b> the minimum viable test (MVT) and make sure to test it with a small set of beta users prior to releasing it in order to validate the initial metrics. </li>
<li><b><u>Decide</u></b> on the proportion of users that will be exposed to the A/B test (e.g. new users, returning users, users who haven't purchased yet, 10% of all users, etc.) </li>
<li><b><u>Run</u></b> the A/B test until the results become statistically significant for the required confidence level (usually 95%). Also ensure that the A/B test occurs during a time period that is considered "usual" activity (e.g. don’t A/B test on a Sunday if users don’t often purchase on a Sunday). </li>
<li><b><u>Calculate</u></b> which version of the test performs better. If the newly tested version is superior, make it the default version of the mobile app and release it into production for all users. </li>
<ul>
<li>If the newly tested version either performs poorly or no conclusion can be reached, record the details and possibly re-assess later. </li>
</ul>
<li><b><u>Observe</u></b> any other tangential effects that the A/B test may have caused such as increased support calls/emails, decreased retention, engineering complexity, etc. It may also be helpful to present some users with a brief survey asking them about their new experience in the mobile app. The results from this survey will add valuable qualitative feedback to the A/B test’s quantitative results. </li>
<li><b><u>Repeat</u></b> the process by running another A/B test.</li>
</ol>
<div style="text-align: justify;">
Ultimately, executing A/B tests is about simplicity and speed. The faster the tests can be run and statistically significant winners declared, the more growth a product will see over time. </div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
The steps given above for running A/B tests relate to users who have already downloaded the mobile app. A/B testing can also be performed on users coming from specific growth channels. Due to mobile's inherently closed ecosystem, attribution is more complicated on mobile apps. However once it is setup correctly, it is possible to track users from specific growth channels so that each channel’s revenue potential can be calculated and optimized.</div>
</div>
</div>
</div>
</div>
</div>
<span style="font-family: inherit;"><span style="background-color: white; vertical-align: baseline; white-space: pre-wrap;"></span></span></div>
Andrewhttp://www.blogger.com/profile/03252749023463514361noreply@blogger.com2tag:blogger.com,1999:blog-1847717784907730054.post-45490612003293779552013-11-11T20:33:00.000+00:002013-12-02T02:27:46.656+00:00Xamarin Features RESAAS Mobile App<div dir="ltr" style="text-align: left;" trbidi="on">
<div class="separator" style="clear: both; text-align: center;">
<a href="http://1.bp.blogspot.com/-KT7O2kvzQXc/UniIuxtEf7I/AAAAAAAAAgA/IUqmvIptUJg/s1600/iPhone+4S+Splash+Screen+3.png" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em; text-align: justify;"><img border="0" height="480" src="http://1.bp.blogspot.com/-KT7O2kvzQXc/UniIuxtEf7I/AAAAAAAAAgA/IUqmvIptUJg/s640/iPhone+4S+Splash+Screen+3.png" width="640" /></a></div>
<br />
<div style="text-align: left;">
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<div style="text-align: left;">
One of the things I am passionate about at RESAAS is our mobile app for iPhone and Android. We are often exploring how our customers use the app differently from the browser experience and then optimizing the experience for that exact use case.</div>
<div style="text-align: left;">
<br /></div>
<div style="text-align: left;">
Xamarin, the company behind the cross-platform development framework that uses C#, recently featured the RESAAS App on their website: <a href="http://xamarin.com/apps/app/resaas_the_real_estate_social_network">http://xamarin.com/apps/app/resaas_the_real_estate_social_network</a>.</div>
</div>
<div style="text-align: left;">
<div style="text-align: left;">
<br /></div>
</div>
<div style="text-align: left;">
<div style="text-align: left;">
I've written previously about our App being <a href="http://www.andrewthompson.co/2013/07/resaas-reblasts-app-featured-on.html">showcased on the Appcelerator Titanium blog</a> (when we used their framework instead of Xamarin) as well as our <a href="http://www.andrewthompson.co/2013/04/resaas-mobile-app-released-for-iphone.html">initial app release</a> back in April 2013.</div>
</div>
</div>
Andrewhttp://www.blogger.com/profile/03252749023463514361noreply@blogger.com0