What do Development Teams and CPUs have in common?
Answer: Relentless Prioritization (from a great PM 🤓) !
Prioritization is the most important discipline that you have to be great at to excel as a Product Manager! If you master this, you immediately impact all other disciplines of Product Management by reducing waste, saving time, pleasing stakeholders and making space for the really important topics.
On the other hand, great Prioritization is also the toughest nut to crack, as this is a multi-variant task with moving targets along the way of solving the rather easy-to-formulate question:
“What is the next task to work on?”.
In this article, instead of deep-diving into the different frameworks (e.g., Prioritization Matrix, RICE, MoSCoW, Kano, YAGNI, …), I would actually want to talk about a small companion that assists our daily-life mostly unnoticed, but has to answer the exact same question within a timeframe of μs (yes! that is “micro-seconds”): I am talking about the CPU scheduler.
Let’s have a look at the parallels between a PM and a scheduler and see if we can learn something from it and take into our daily business.
The CPU scheduler
The scheduler is a tiny program that sends jobs to the CPU (central-processing unit) and is built into every digital hardware that has to calculate multiple jobs at the same time. Be it to decide to update my watch or to show a push notification, to either check for incoming mails or to update the text that I just entered on my … <> … keyboard.
BTW: If you think the above examples are not important enough, please have in mind that a CPU scheduler is embedded in the Autopilot of a Tesla and decides every day about life and death of many people.
You could say that the scheduler is the Product Manager of the different cores of a CPU that form the development team. The stakeholders of a scheduler are the multiple processes on a machine that usually have each different priorities on their timely execution.
But… Can’t we just add more…?
A natural reaction to a prioritization question is usually to add more units of the scarce assets. This could be number of cores of a CPU as well as human beings in a development team.
In both cases, the answer to the question whether to “just add more” is always the same:
“Yes, but …”
Going with Brook’s law, adding more personelle to a project does not actually make it faster. Why is this? Well, there is communication overhead added to each process which eventually (when adding even more people) will lead to a downtrend in productivity of the whole team.
The same applies for computers and CPUs: The more cores you add, the higher is the need for specialized software to actually make use of the different cores (read: the more overhead you need to invest to build the system right).
And even if we would ignore the problem with the added overhead, we would still have a scalability issue, as adding more persons/cores scales linear for both, workforce and costs.
So, nothing to learn here from the CPU, but interesting to see that even in this facet, we can draw parallels.
KPIs and Optimization
Before starting to discuss different ways of optimization. It is wise to first define the KPIs that you want to optimize for. For the scheduler, this boils down to one of these three:
- maximizing utilization (PM: “was everyone busy?”)
- maximizing throughput (PM: “how many tasks did we complete?”)
- minimizing turnaround time (PM: “how long until everything was finished?”)
As you can see in the verbalized translation, the optimization criteria can be found in our working world. While all three of them are pretty self-explanatory, it is crucial to understand that you have to choose the target KPI wisely.
The outcome of prioritization (and hence the tasks that will have been worked on) might differ tremendously, depending on the choice you made.
To give just one example from my past experience: In my early days of my life as a PM, I thought my job was to keep every person in the team busy (maximize utilization). However, I soon realized that this actually lead to poor product decisions on a global product scale, as some tickets had to be created just to keep the iOS developer or the analyst in the team busy. That eventually lead to a feature creep, as I included items on our development board just for the sake of keeping everyone busy. It eventually also slowed down the whole team, as this lead to an unfocused team and poorer code-quality.
On the other hand, if you only look to optimize for outcome of larger epics, this might lead at some point to the situation that individuals feel bored, as they cannot participate actively in the current development fields.
Types of Scheduling Algorithms
Let’s have a closer look at different algorithms a scheduler might be implemented with. In the end, we are here to learn something and for each of the algorithm, I will draw a connection to our daily-life as a PM in form of a takeaway.
There exist two types of schedulers preemptive and non-preemptive algorithms:
In preemptive scheduling, tasks have priorities and the scheduler can assess whether to stop a currently running task and execute one with higher prio, or not. Sounds familiar? 🙂 Ever heard of shifted goals or a critical production bug that appeared and crashed your beautifully planned Scrum-sprint?
On the other hand, there is non-preemtive scheduling which is more general-purpose, but on the other hand, pretty easy to follow. If the CPU is currently busy with a long-running low-prio task, it will only start on the quick-and-high-prio one, after the first is finished. Needless to say, that while this is clear to formulate, it might not lead to the best performance — especially if you favor outcome over output.
Depending on the flight level that you are working on, a non-preemptive scheduler is comparable to a Waterfall planning approach which is rather static, whereas a preemptive technique feels more agile, as it can adapt more quicker to changes in the prioritization.
In our world of Product Management — as we are smart people — we mostly look at preemptive scheduling algorithms, as the basis for our prioritization.
However, this brings all sorts of complications into the prio game.
Algorithms
Let’s have a look at the different algorithms that exist in CPU schedulers and their impact towards our three main KPIs:
First Come First Serve (FCFS)
This is a non-preemptive algorithm that is pretty easy and simple to implement, as the scheduler simply forwards every task to the CPU the way they entered the queue.
👩💻 PM-Parallels: This is comparable if you’d ignore all priorities of each task and simply add them to the backlog without the possibility of resorting.
Effectiveness: Well, this definitely keeps everyone busy, it is questionable if this yields the best outcome, as high priority tasks could end up waiting in the back of the queue and once they arrive in the beginning, it is too late for them to be executed.
👩💻 Takeaway for a PM: If you’d apply this technique in your working live, it would be easy in terms of prioritization, but soon become rather hard for you to manage your stakeholders’ happiness. Bottom line: Do not use this if you care about the product and your job!
Shortest-Job-First (SJF)
In this algorithm, the scheduler would need to know how much each task in the queue would need until their execution is finished. The task that has the shortest time to be finished will be assigned to a CPU. This could be either preemptive or non-preemptive, given the implementation. A special version is Shortest-Remaining-Time-First (SRTF) where you stop a process at some point and take the remaining time to finish this job into account for the sorting.
👩💻 PM-Parallels: Given that you know for each task how long its implementation takes (e.g., using Planning Poker), this is a stable sorting for your backlog.
Effectiveness: In an ideal situation, where each ticket delivers the same value as the others to the users, this would be great to allow for a faster time-to-market. Other than that, definitely a good way to keep everyone in your team busy.
👩💻 Takeaway for a PM: I have yet to find this setup where on the one side, the team gives stable estimations for each ticket and at the same time, each ticket delivers the same value to the users. Without those prerequisites met, this will probably yield to a product where you only focussed on the small things rather than what is best for your users. Bottom line: Do not use this in the real-world but maybe tweak it a bit!
Round Robin (RR)
Round-Robin scheduling tries to give every incoming thread the same share of CPU time. This can either be implemented with a time-based approach that cuts off running processes if they ran long enough, or it can simply take the next task from another source, once the active one is finished.
👩💻 PM-Parallels: If you have a lot of stakeholders and want to please them in roughly the same ratio, this would be one approach to do so.
Effectiveness: Not much, as this involves a lot of context switches which again take time to save the context, restore it again. Also it will not pay into our three main KPIs defined above.
👩💻 Takeaway for a PM: As I said, good if you want to please every stakeholder from time to time. However, you should rather rank your topics by user value and outcome for the user and not by the stakeholder Bottom line: Only use this, if you really have no clue what the outcome of each ticket is!
Priority-based (PB)
Well, you can guess how this works. Each task is given a priority. The scheduler adds the task to a sorted list where it fits, based on the given priority. By that, a high-prio task can overrule other tasks that stayed already in the queue for quite some time.
👩💻 PM-Parallels: This is probably the closest algorithm to how we work in our daily business life. However, you should take into consideration, that using this on a set of heterogenous tickets could lead to a maximum of throughput but also a lot of context switches, if you choose a wrong priority that mixes up different tickets of epics.
Effectiveness: Pretty good, but only depending on the level of certainty that you can define the priority of the ticket. As a rule of thumb, the priority should be a function that takes the outcome for the user into account.
👩💻 Takeaway for a PM: Given that you invest a reasonable amount of time in prioritizing the tickets, this will produce not only a value-driven result that you can defend to your stakeholders and team members, it also brings clarity in your working life. Bottom line: Do use this!
Multilevel Queue (MLQ)
In this approach, you predefined different queues, based on properties that you choose. A classic approach is to have two queues (foreground, background) for tasks that either have to be handled directly (foreground) or are of lower priority (background). Once a process is in one of the queues, it has to stay in this queue.
👩💻 PM-Parallels: This is something that I would see on a higher-level setup with many teams working on different topics — where each of the teams then has one queue to work on. Though you could also think of a “fast-lane” approach inside a team for very important bugfixes that are then to be put in the respective queue.
Effectiveness: The effectiveness of this approach lies within the amount of tasks that come for every queue and whether or not one queue is drained
👩💻 Takeaway for a PM: If you have a lot of categorizable tickets with a steady inflow, this could be useful for you to incorporate. If implemented wrong, this could quickly lead to drained queues and bored team members. Bottom line: Use with caution!
Conclusion
So, now that we looked into some of the algorithms that a CPU scheduler is implemented with, what can we take with us? There are certain approaches that I find interesting to incorporate into the daily life (i.e. PB and MLQ). However, each one needs adjustment and also does not work out-of-the box. You still need to do ground work with your team together to estimate tickets, cut or merge where needed and estimate product value for each topic.
Long story short
I presented the parallels in the working life of a Product Manager and a CPU scheduler. We then explored different scheduling algorithms and learned which of those make sense to incorporate into our work and how to do this.
Happy prioritizing! 🤓
Discussion
I am very interested to hear your thoughts and experiences in prioritization and if you found this article worth or not.
- Could you take something with you from this article?
- Anything that you would like to add?
- Any fun war-stories that you want to share?
If you are interested in connecting with me, please do so and please drop me a short line where you are coming from.
