Background
OpenWISP is a powerful suite of software tools that makes network management easier for administrators. While many people benefit from OpenWISP-powered networks, they don’t see OpenWISP itself; it’s the network administrators who interact with it directly. Federico, one of the creators of OpenWISP, wanted to understand how these administrators were using the tool.
Because Free/Libre and Open Source Software (FLOSS) can be used without any direct contact with its creators, it can be challenging to know who’s using the software and how they’re using it. This is particularly true for OpenWISP, which struggles to determine how many deployments are out there and what versions are being used.
Without knowing who is using the software and how it’s being used, it’s tough to figure out how to generate sustainable income for the project. Relying solely on administrators to reach out for support only gives a partial picture. Before implementing Clean Insights, Federico’s understanding of OpenWISP’s user base was limited to support requests from the few administrators who contacted them.
Motivation for Implementing Clean Insights
OpenWISP needed to start collecting usage metrics. This was something they felt they should have done a long time ago and when they started working on SUSTAIN, the funders insisted it was one of the main goals.
Objectives and Questions
The questions are:
- How many people deploy our software?
- How many of those deployments survive over time (how many users give up)?
- How many people keep it up to date over time?
Implementation Process and Challenges
The process has been iterative; they started with a simple implementation and then improved on it until they were finally able to answer the questions mentioned earlier. The main pain points have been:
- Getting the data right, as they initially had duplication issues.
- Working with the event schema, which felt a bit limiting. Finding a way to create meaningful charts with the data was challenging, but they succeeded in the end.
- Collecting low resolution data rather than highly precise measures which can be accidentally identifying and potentially harmful.
Approach to Implementation
They chose to implement on their own because they felt the current available implementations wouldn’t allow them to answer the questions they were looking to answer in the way they wanted.
Data is collected at three points in the user’s process:
- Install: when OpenWISP is installed the first time
- Upgrade: when any OpenWISP module is upgraded
- Heartbeat: once every 24 hours (heartbeats tell us if an OpenWISP instance is alive. They are used to gauge the number of living OpenWISP instances)
Impact on Development and Decision-Making
At the moment, there haven’t been any changes to OpenWISP. The team feels they need to collect more data points before making any informed decisions.
Metrics and Data Insights
They collect data on OpenWISP usage to gauge user engagement, satisfaction, and upgrade patterns. This informs development decisions, ensuring continuous improvement aligned with user needs.
The openwisp-utils module includes an optional sub-app openwisp_utils.metric_collection, which allows collection of the following information from OpenWISP instances:
- OpenWISP Version
- List of enabled OpenWISP modules and their version
- Operating System identifier, e.g.: Linux version, Kernel version, target platform (e.g. x86)
- Installation method, if available, e.g. ansible-openwisp2 or docker-openwisp
The data above is collected during the following events:
- Install: when OpenWISP is installed the first time
- Upgrade: when any OpenWISP module is upgraded
- Heartbeat: once every 24 hours
Statistics are currently enabled only on the development version. They are working on finalizing the release blockers so they can publish the new version and get the stats out in the wild.
Crucial Design Decision: Opt-out available at anytime
In the system info page, superusers can easily opt-out from the Metric Collection at any time. The link “why we collect metrics” leads to OpenWISP Usage Metric Collection.
You can opt-out from sharing data any time from the “System Info” page. Alternatively, you can also remove the openwisp_utils.metric_collection app from INSTALLED_APPS in one of the following ways:
- If you are using the ansible-openwisp2 role, you can set the variable openwisp2_usage_metric_collection to false in your playbook.
- If you are using docker-openwisp, you can set the environment variable METRIC_COLLECTION to False in the .env file.
Next Steps
We will continue to keep in touch with the OpenWISP team and hope that once more data rolls in we can provide an update to our impact report with key findings and learned insights which resulted in assisting the OpenWISP team make crucial decisions or updates to their administrator console.
“Going through this process has been really useful and I hope to collect more data gradually over time.”
–Federico, OpenWISP founder & developer