When building a knowledgebase, each data point individually is not very valuable, but it in aggregate, there is a data Network Effect, where each data point that is added, increases the value of the whole. If one data point is missed, most people won’t notice, but if the knowledgebase is not updated regularly and urgently, that will degrade trust with users. So, despite some aspects of updating the knowledgebase taking quite a bit of time, the work is still urgent and important.

Recently my team has been working on a project to add generative AI to some processes that are pretty time consuming but individually not very high value for our team.

In order to investigate how we would introduce generative AI, the team began testing a few prompts and tweaking them to see how well they performed at the tasks we assigned. We had some pretty early successes, which was great, but this is where we starting to bump up against some fundamental Agile principles.

How we went off track

With our early success, the team wanted to start sprinting for the end zone, identifying a number of tools they would need access to in order to create a full end to end solution that would have the highest possible performance and the best possible results.

One of my favourite things about working with a team of passionate and highly intelligent people is that they are exceptionally good at picking up new skills to discover solutions to new problems. I love the pace of their brilliance, but we always need to make sure we’re staying in alignment with other teams and the other priorities of the business.

In this case, we went out of alignment when the team requested access to many new tools, and a certain amount of software developer time to implement a complete end to end solution. This is where the alarm bells started to go off for me.

As the manager of people so inspired to solve a problem, I needed to keep our plans in alignment, while also not closing off the possibility of taking this path in the future.

Keep the door open

First I asked my team which of the tools would be the most useful and why. They gave me a complete breakdown of how all the tools fit together in a workflow, and points where additional tools could be added in the future.

Next, I asked how much of a hindrance it would be to try to build this generative AI workflow, using only the tools we currently have access to. They took a few minutes to think, then started modifying the workflow. It wouldn’t be a fully end to end solution that would spit out an answer for the curators to review. We’d have to copy and paste here and there, upload some files, and press a few buttons, but that could still represent significant time savings for us, without compromising any human review.

I pitched it to the team as hacking together a small proof of concept, that we could show to senior leadership to justify the investment in additional tools and software developer time. By keeping it Agile, we could quickly test our theory, run experiments, and present it to senior leadership before investing significant resources. This way, we wouldn’t need access to all the tools and create a fully scalable solution before we were even sure it would work. A light investment at the beginning allows us time to validate the idea before we consider scale and all the other bells and whistles.

One risk with taking the time to run a proof of concept test, is that the team may become frustrated with the pace of progress. We can avoid this risk through good product management. While one project may move a little slower, but still in alignment, the product manager can talk with more users to see what other features could be built, and run more quick proof of concept tests with the team. Through this mechanism, you can discover more, test more ideas, and only invest in the ones that have the best chance of solving a user’s problem in a way that aligns with the company’s strategy.

Wrapping Up

When building anything, especially something the team is so excited about, be sure to do it in smaller steps. A simple proof of concept can validate an idea and justify the investment of further resources. Running straight to a complete product risks bringing your team out of alignment with other teams and the organizational priorities. By taking the time to build a quick proof of concept, you also unlock the ability of your team to run more experiments while dedicating more resources to the most promising results.