.jpg){kind=small}
Why do traditional approaches fail?
When developing new knowledge-management systems (KMSs) most teams take a ‘waterfall’ approach. This approach follows a linear path from requirements gathering, to design, development, testing and implementation.
Despite being seemingly intuitive, waterfall development has fallen out of favour due to its tendency to increase costs and delays. More importantly, waterfall's one-and-done approaches usually fail to fully address users’ needs and requirements. Some teams use Design Thinking to better understand their user requirements. However, Design Thinking workshops used within a waterfall framework will still fail for the reasons outlined below.
A better approach.
How can I address my user’s needs, without adding delays and costs to my project’s development?
How do I ensure my KMS will have great uptake and usage even when I don’t have good visibility into the team’s evolving needs and what content to prioritise?
The answer lies in continuous improvement practices and iterative development — put simply, placing the power in your users’ hands to decide what features to develop over time.
Continuous improvement processes are powered by feedback loops. These feedback loops highlight incremental improvement opportunities and reflect, in real-time, what your users want.
By creating a feedback loop in your KMS:
- development planning is informed by data and feedback from your users;
- new features and improvements can be developed according to this feedback; and
- user engagement and satisfaction can be determined quantitatively against previously collected data and feedback.
How could feedback loops apply to my KMS?
✅ Site analytics ✅ Page heatmap ✅ Search analytics ✅ Voting and feedback forms
Creating a successful feedback loop for your KMS depends largely on how you are gathering user feedback and data. It is important to consider what metrics you would like to gather, and how these might be useful to you when planning for future development. This may include quantitative data such as usage statistics (which can be captured automatically within the system itself), or qualitative data such as direct feedback from users (which requires active effort and participation from users). Using this data, you can form a hypothesis on what further development is required and justified - which would ultimately lead to improvement in user satisfaction and uptake in usage of your KMS.
Here are some of the best feedback tools to deploy on your SharePoint knowledge base.
Site Analytics
One of the best ways to collect quantitative data on how your KMS is performing is to use site-wide page analytics. These stats provide insights into your most / least popular pages, files, and content.
You might use this data to determine where your users spend the most time, how often they browse your KMS or what information they frequently refer to. Your feedback loop can also use this data to determine whether changes made have had an effect on the usage or popularity of changed pages. You can implement site analytics simply by using Sharepoint’s built-in analytics (pictured above).
Integrating Google Analytics or Microsoft Clarity into your KMS takes this a step further. These integrations allow you to gather more fine-grained analytics about your users’ behaviour and devices used to access your KMS. Detailed analytics can be used to cohort your users into power, casual and new users. These cohorts then help you to determine the type of pages or information that specifically appeal to new users, or features that could be implemented to encourage your casual users to become power users.
Page Heatmap
In addition to page analytics, tools like Microsoft Clarity allow you to collect page ‘heatmaps’ and session recordings that highlight and replay where your users are clicking and scrolling on your site.
This can help you determine where and when your users are interacting with a specific element on a page e.g. a calendar or newsfeed; or whether content that requires more scrolling down is reached by most users.
You can implement page heatmaps using Microsoft Clarity (pictured above), or a third-party solution such as Matomo.
Page heatmaps can be combined with page analytics to gain an even deeper understanding of your power, casual and new user cohorts, and how these cohorts interact with your KMS.
Search Analytics
Analytics can also be collected that detail search terms and patterns used by your users to discover information on your KMS. This quantitative data can include metrics such as:
- total search volume;
- result type (i.e. article, file, page);
- top search terms; and
- abandoned searches.
You can implement search analytics simply by using Sharepoint’s built-in analytics(pictured above).
Search analytics data is highly valuable for informing your feedback loop. For example, your KMS might contain content requested by your users. However, your search analytics data demonstrates that your users are unable to find this content — possibly due to using search terms that were not contemplated when creating the content. This suggests that you should improve the searchable metadata for the difficult-to-find content.
Abandoned search data is also useful to clearly illustrate what content your users have been searching for and could not find on your KMS — providing you with strong justifications for creating this new content.
Voting and Feedback Forms
In addition to the data collected above, you can also request feedback directly from your users. For example, at the bottom of your search results page you may include a text-box enquiring as to whether your user found what they were looking for. This provides your users the opportunity to tell you directly what content is relevant to them and should be included in your KMS.
You can implement feedback forms using the built-in Microsoft Forms Sharepoint web part (pictured above).
This feedback form can be extended by providing users with the ability to ‘vote’ on suggested new content or features. This empowers your users to be involved in decision-making affecting your KMS and can help to determine low-impact suggestions.
Voting functionality can be easily implemented using Sharepoint lists.
You should be careful to avoid overburdening your users with opportunities for feedback and survey collections. It is important that these opportunities are presented in the right context and in a way that requires minimal effort for users to provide input.
You should also be wary of selection bias creeping into your results. This bias alludes to the general observation that users with strong (typically negative) feelings or free time will engage with feedback opportunities (see Qualtrics article and Atlassian article for a summary of sampling and response biases in user feedback surveys).
Putting it all together
With the above tools, you now have a series of feedback loops that help provide insightful quantitative and qualitative data about your users and how they are interacting with your content.
⚠️ Case study example: Improving document discovery in a KMS
To demonstrate the efficacy of feedback loop’s on KMSs, here is a fictitious case study concerning Red Umbrella Co.
ℹ️ Context
Red Umbrella Co is a legal services company providing services to clients on behalf of other law firms. Red Umbrella has developed a KMS using Microsoft Sharepoint which stores all company policies, process documents, know-how guides and FAQs. The KMS allows employees to self-serve most IT needs related to their day-to-day work without submitting support tickets to the IT team.
❗Problem
Through feedback forms within the KMS, Red Umbrella has received feedback from users that they are unable to locate a document with instructions for setting up 2-Factor Authentication (2FA) when ‘working from home’.
Investigating this issue further, Red Umbrella gathers data from their search analytics which demonstrates that users have been searching for the 2FA document — however, these searches become ‘abandoned’ after users are unable to locate the relevant document.
Additionally, page analytics for the 2FA document illustrate that it has been rarely accessed since creation — particularly when compared to similar support documents receiving much higher traffic.
❓Hypothesis
From the data described above Red Umbrella forms a hypothesis concerning the problem.
Hypothesis: Users are not accessing the 2FA document because they are unable to discover the document using navigation or search functionality within the KMS. Traffic to the document can be increased, and negative feedback reduced, by improving the document’s placement in the navigation hierarchy and adding additional search term metadata.
🔨 Improvement
Following this hypothesis Red Umbrella plans to develop two improvements to their KMS:
Document placement in the navigation hierarchy
The 2FA document is currently stored deep within the policies section under IT department tab. To make navigation more intuitive, an additional link to the 2FA document is added to the policies section under the ‘working from home’ tab in accordance with users feedback.
Search term metadata
Metadata and tags associated with the 2FA document contain limited keywords and terms including: two-factor authentication, VPN access, information security and account management.
Based on search terms used by users in ‘abandoned’ searches the following metadata search terms are added to the 2FA document: passwords, logging in at home, access from home, working from home login and accessing account from home.
✅ Review
After implementing the improvements above, Red Umbrella re-engages with the feedback loops in place to determine if their hypothesis was supported and if the changes were successful.
The page analytics data show that a noticeable increase in traffic to the 2FA document has occurred. Additionally, ‘abandoned’ searches containing the new metadata terms have declined. Demonstrating that users have been able to locate the document they were searching for.
Finally, Red Umbrella no longer receives negative feedback from users concerning their inability to locate the 2FA document within the KMS.
Using simple yet powerful feedback and data, Red Umbrella have improved their knowledge base, and in turn, have made their employees’ lives a lot easier.
--
If you are interested in the case study above and would like to integrate some of these feedback loops into your knowledge-management systems, visit our website at TILT Legalor reach out to our team members Cooper Corbett or Isaac Wong.