Does embedded software fit with remote work?

Home offices installed for 140 people in less than 24 hours, software deliveries progressing (almost) as planned, development activity that continues unabated, and customers who are playing along.

What if this extraordinary time were to ultimately shake a few assumptions about R&D activities in embedded software?

What seemed impossible or far-fetched a few years ago is now essential for the survival of many companies in the embedded sector world, namely, working on complex innovation projects with distributed hardware and engineering teams. Finally, within 24 hours, all of Witekio’s teams converted to a new work organization that could have taken 12 months to set up in other circumstances.

So, what have we learnt at Witekio?

We want here to share our experience, tips, and our doubts in the hope that it will be useful for the wider embedded software development community.

To be clear, at Witekio we have long talked about the possibility of regular remote work for our engineers, but we had our doubts. Primarily this was because we believe strongly that human contact, easy access to expertise, and informal exchanges in the office are key in the innovation phase of software projects. Additionally, our work is far from being totally digital. In fact, a lot of the charm of embedded software developments lies in the profusion of electronic cards, cables, machines of all sizes, routers, oscilloscopes, probes, and other objects spread across our desks.

In short, with the goal of maintaining a close-knit team, high quality of work, and operational simplicity, we have remained focused on production on our premises in our five locations in Europe and North America, while carrying out preparation for the reorganization of our work in order to make us more flexible in the future.

This preparation, it turns out, is what has allowed us to react so quickly – even to the point of anticipating various government decisions – to ensure the continuity of our services to our customers.

5 Challenges for Witekio’s Remote Work Organization

Moving an entire company from on-premise to remote in just a few days didn’t come without challenges. Indeed, we identified five major issues associated with this new remote work organization being, from the general to the specific:

1. Hardware management
2. IT management
3. Software production routines
4. Modes of communication
5. The Witekio Culture

Here are the challenges we faced, and how we sought to address each one.

Challenge 1 – Hardware Management: The Art of Distributed Logistics

All of the projects we work on involve a physical product, with the degree of progress in its development impacting the level of access available to an engineer.

In some cases, the optimization of an existing product may not require any hardware since the device would have a sufficient level of connectivity and stability to be worked on remotely, that is if no physical access is required for testing. In other cases, the early development phases of new products, engineers work on prototype boards, evaluation boards, or debugging mode boards that require a direct physical connection via Ethernet or another specific port.  In such cases is it not only necessary to have the equipment close to hand but also access to specific connectivity equipment like routers, switches, and cables that may not be common at home.

What’s more, without rigorous management of the equipment, it can be very easy to lose track of where the equipment or the test and measurement tools associated with each phase of the project are. Such problems mean losing precious time in the execution of a project due to a simple logistics problem.

With such issues in mind, here’s what we have implemented at Witekio:

• All equipment moved off-site is referenced in a tracking tool that is updated on a weekly basis as it moves;
• Equipment delivery times are now factored into the planning of our commercial proposals and deliveries;
• We have established the role of ‘Internal Courier’ to pick-up and drop-off equipment, with this person the sole employee duly authorized by the company to travel;
• Since 2017 we have established a “farm” that hosts certain cards/products that allow remote access, compilation, and testing – read about it here;
• We aim to have a greater-than-normal stock of connectivity and test equipment than we would have when everyone is in the office, including switches, routers, oscilloscopes, probes, and more – we have even asked our customers for access to additional equipment that belongs to them.

Challenge 2 – IT Management: Online and Self-Service Solutions

We are a fortunate company in that we have staff that are very computer literate. A direct consequence of this computer literacy is that our IT team does not often need to provide help for basic issues including Internet connectivity, software installation, or cabling. Witekio’s approach to IT management has always been to offer all of the team a set of tools to work with, with a fairly flexible framework as to which they use and a strong self-service approach.

In addition to this company-wide approach, each department can both choose and manage its own tools in order to avoid the excessive centralization of very specific business tools. For several years now, we have operated a decentralized and fully SaaS approach to all the company’s tools. Email, HR, CRM, ERP, hosting and software delivery management, project and ticketing management, knowledge base, finance and accounting are all decentralized and hosted in the cloud.

All that remained centralized was the documentation base that was, when COVID-19 struck, in the process of being migrated to a SharePoint solution, with the exception of files that the company considered too sensitive or that were too large to be supported on SharePoint. It’s no surprise to report that this migration has been accelerated over the last two weeks.

In essence this means there is no functional difference between the home and work IT environments of our teams, no limits to productivity, and even easier collaboration thanks to native team-oriented tools.

Building on this, here is what we have implemented to ensure there are no issues on the IT management side:

• Fully decentralized SaaS enterprise software to be entirely digital and establish non-location dependent IT;
• An IT department that is there to provide a set of tools to manage the security policy, and not to inspect users for their adherence to it;
• Native collaborative tools to enable work on deliverables by several people simultaneously.

Challenge 3 – Software Production Routines: Rigor and Collaborative Tools

Witekio has set up numerous routines to ensure consistent quality in the execution of our projects, with daily meetings, planning poker, sprint reviews and more.

The fact that we are now all remote adds some new constraints to these routines. Our teams now have to manage busy conversations between teams, make themselves understood without easily being able to draw on a whiteboard, handle a physical device, or present a deliverable on a physical product via a simple computer camera. There is also the problem of replacing the informal but so important pre- and post-meeting moments, as well as the need to keep calm in the face of the multiple technical videoconference concerns.

With these issues in mind, here is what we have implemented to ensure our routines remain efficient and effective:

• Updated our ‘Cultural Manifesto’ with specific items related to remote work such as an order of speakers, a ‘Definition of Ready’, and numerous ‘Do’s and Don’ts’;
• Extended daily meetings to 25 min to allow more time for informality;
• Adopted a collaborative tool (PokerBot for Slack) for simultaneous voting during planning poker;
• Dedicated a webcam for the presentation of progress towards the final product to avoid juggling with a PC camera.

Challenge 4 – Modes of Communication: Organization to Avoid Saturation

Slack, Microsoft Teams, email, and the humble phone are all great tools to maintain communication, but all can also lead to decreased productivity and information losses as their use proliferates.

In order to avoid negative impacts, guidance rules on the purpose and rules for each communication channel need to be defined. Occasionally differentiated uses can emerge naturally, and it can sometimes be enough to simply write down and share these unspoken rules. Where this doesn’t occur, though, it is up to management to define simple and shared principles.

Faced with this challenge, here is what has been decided at Witekio:

• Slack is reserved for informal group discussions – primarily IMs – and organized by topic or team. Slack can be used for internal conference calling, too, even if the video quality leaves something to be desired compared to other solutions. Witekio has asked for all responses to be made in individual ‘threads’ where appropriate to avoid the alternative long list of exchanges in a ‘channel’;
• Email is reserved for anything that requires reading, review, and a formal response, either internally or externally;
• Microsoft Teams is preferred for internal and external video communications;
• The telephone remains an option for informal discussions or for the clarification of topics that could easily require the exchange of dozens of emails, in particular with customers and external collaborators.

Challenge 5 – Working the Witekio way: Maintaining Focus, and Shared Objectives

Being alone at home all day and for several weeks can lead to a loss of sense of community, a loss of motivation, and a lack of rigor as home is usually associated with activities other than best-practice embedded software development. In addition, last-minute cancellations or changes in the expected chain of events, appointments, and project milestones negatively impact priorities and lead to frustrations all around:

What does it matter?

Why make an effort if everything is going to be canceled or changed anyway?

Is there still a pilot on the plane?

To avoid this, we felt it was important to intensify all team management routines and to replace the informal feedback that might be gathered around the office with daily slots where everyone can share their issues, objectives, and any problems.

These daily slots also provide a way for managers and project leaders to motivate teams, offer them advice and perspective, and share company news. While this does, in turn, lead to a significant overload of meetings and a certain amount of time that is not directly productive, it is more than balanced by the capacity to maintain the invisible but real bonds of shared goals and a common culture.

Specifically, here is what we have implemented at Witekio:

• Daily meeting for all departments, including support staff;
• Daily (instead of weekly or even monthly) monitoring of all production indicators including occupancy rates, project burn rates, and any possible slippage;
• Weekly ‘virtual coffee meetings’ in every office and on a voluntary basis in order to exchange on non-professional subjects and maintain personal ties;
• Weekly emails from top management to provide business information and corporate updates, generally concluding with a hangout session;
• Encourage all initiatives that create bonds and help maintain morale, whether through Slack channels dedicated to humor, creating a responsive monoboard, or sharing everyone’s working conditions with the hashtag #balancetonsetup.

Many Unanswered Questions Remain

Almost certainly we will have more adjustments to make in the coming weeks, and much more learning to share. Indeed, setting up an organization in a crisis situation such as that forced upon us by COVID-19 has allowed us to advance very quickly thanks to a shared sense of urgency. At the same time, though, it has meant putting to one side important topics that must be addressed if Witekio decides to continue remote work as part of a possible operating procedure.

Even after only a couple of weeks we anticipate additional challenges that remain, for the moment, important if also non-urgent:

• Engineers working permanently from home will need a better setup than many that have been implemented quickly to and only to handle the present crisis;
• Confidentiality constraints requested by our customers could sometimes be limiting on Witekio, if only because remote work has not been foreseen;
• Difficulties in terms of office management could emerge as employees alternate between remote work and being present in the office; indeed, an office that is empty half the time is sub-optimal in terms of the management of our physical premises;
• Difficulties will emerge if we are forced to move equipment several times a week as this creates the potential for loss or damage, will create more complex project challenges to be addressed, and therefore imply additional costs or reduced margins;
• Managing a mixed remote/on-premise team poses challenges as gaps in terms of access to information and a kind of distance between remote workers and their on-premise counterparts both emerge;
• In the long run, as the probability of project changes for engineers increases and with those changes come more logistics flows to manage, there is a greater risk of deterioration, more complex project changes to implement, and extra costs to be absorbed;
• Remote work brings with it an overload of synchronization meetings, especially for managers, which could in the medium-term lead to serious performance and workload problems.

These points aside, while this COVID-19 crisis is obviously dramatic from a health, economic, and social perspective, we are also trying to see it as an opportunity.

From a work organization point of view from within a sector where there are considerable constraints, this unpredictable crisis allows us to test new models of work, to improve collectively our understanding of subjects as varied as management methods, technological infrastructure, software production organization, knowledge management, and knowledge sharing.

And what have you implemented for your R&D teams? Share your experiences with us!