Developing commercial software for untangling sound - 5 questions with Simon Harvey

Simon Harvey is the Director of Software at AudioTelligence. Simon has almost 20 years of experience developing and leading engineering teams, working on high performance software solutions across a range of applications. Simon has been responsible for the successful delivery of multiple customer projects for AudioTelligence.
Tell us about the software you’re developing at AudioTelligence

At its heart, the aiso™ software is based around the idea of untangling sound: breaking life’s soundtrack back into its individual parts. That’s a hugely powerful capability that has applications from hearing assistance to transcription, voice control to making a video. It’s that breadth of deployment that makes software development so interesting at AudioTelligence.

Not only do we implement the software across a large variety of platforms (from custom embedded OS to Android, Linux and Windows), but the computing capabilities of the devices can vary by several orders of magnitude. This is one of the reasons why we use C++ extensively: it’s highly portable and very efficient, extracting the best possible performance with the lowest CPU and power costs, whether that’s at the edge on a smartphone, or at scale in the cloud.

Good signal processing code with minimal computational complexity is one facet of what we develop, but we also straddle multiple layers of the software stack. Unlike cross-platform, hardware-agnostic graphics frameworks, real time audio processing approaches can be more fragmented, including custom hardware and very different operating system support for audio handling. This means that when we want the very smallest latency (e.g. for hearing assistance applications), we could be drilling into the lowest levels such as I/O hardware and thread scheduling. At other times, we might be looking at the middle layers: how APIs expose microphone handling. For the uppermost levels, we’ll be tackling systems integration problems; perhaps coupling the audio handling to external computer vision or automated speech recognition solutions.

We span the full software development lifecycle from prototypes and demos to the monitoring and maintenance of production code.

Tell me a little bit about your role at AudioTelligence?

There are a number of software engineering teams at AudioTelligence. Each team could be working on core code or be developing solutions for specific projects. My role is to coordinate activity between those teams, ensuring that they’re not reinventing solutions and that they have the support they need from test engineers and researchers.

One of the challenges for a smaller company with a big technology is keeping focus when there are so many opportunities. One of my functions is to minimise the cost to our customers by re-using code and recognising technical overlap, rather than wastefully throwing resource at a project and creating multiple solutions to the same problem

The other side of that coin is talking to customers and working with them and our commercial team to ensure that the feasibility and cost of a project is accurately estimated from the outset and regularly refined during its course. Sometimes two ostensibly similar projects can turn out to have very different technical challenges; equally two very different customers can end up with very similar core technology needs.

The commercial and engineering teams are very well supported by our research team who design our audio processing algorithms. Part of my role is to provide a bridge between them and engineering so that feedback on real-world performance can travel in one direction and improvements and novel solutions in the other. It’s only by considering the requirements of multiple projects that we pose the right questions to research. In particular, there are often very different solutions for higher-vs-lower latency or where there are time-space computing tradeoffs.

I also try to facilitate the various engineering teams’ work; looking for pain points such as automated testing, build systems or issue management and ensuring that they get the DevOps support they need.

Which software development methodology does your team use?

I’d describe it as “pragmatically agile”. Some customers are happy with weekly or daily interactions, but not all customers engage with software development in the same way and some aren’t in a position to offer the rapid feedback and iteration that fuels Agile so effectively. Additionally, prototype or “green field” projects are often hampered by requiring a significant amount of effort before there’s a tangible implementation through which realistic commercial requirements can be gathered. Nonetheless, we develop to two week sprints, with both internal and external product managers and stakeholders shaping the backlog of issues.

We operate CI/CD across all our supported platforms and architectures, though, here again, there is a pragmatic balance between the low integration cost of maintaining a singular branch versus the stability of release branches. Our target devices are often consumer electronics products and not always as automate-able as we’d like, so there’s almost always value in supplementing CI tests with some manual testing.

Software solutions

In terms of percentages, how much of a developer’s time is allocated for various tasks?

It’s difficult to define as the balance shifts during the course of a project and there are always projects at different stages. There tends to be more support required at the beginning and end of a project but there are some constant activities such as designing and writing unit tests for new functionality, collaborating with Research to define and port new algorithm algorithms or working with Test to validate implementations.

We invest a fair bit of time maintaining CI and CD systems so that the software remains robust across our wide ecosystem and there aren’t too many surprises when integrating different teams’ work. In a similar way, we also make sure we allocate a significant amount of time for refactoring to make sure technical debt is managed and that the software is able to support evolving requirements. This is never done “in a vacuum”, but in response to actual challenges encountered during software delivery to customers.

What, in your opinion, are some of the most important qualities for a successful software engineer?

I think you have to be humble. Good engineers start from the assumption that they’ll get something wrong. They question their understanding of requirements, they use every tool they can find to catch their mistakes when developing the code, they’ll make sure the code continues to check for mistakes when running and they’ll log and monitor real-world behaviour after the software is released.

Relatedly, one of the best things you can do as a developer isn’t to write software, it’s to use it. It’s harder to write bug-free software than many developers expect, harder still to prove it’s bug free unless it’s been widely used for a long, long time. Where licences permit, be humble enough to see if someone else has already worked on a solution for a similar problem.

Attention to detail is very important. Everyone’s heard of major bugs or security vulnerabilities whose root causes turned out to be a missing brace or misplaced double quote, but I’m also talking about noticing when something doesn’t seem quite right and being aware of implicit assumptions.

Developing commercial software is a collaborative, creative process, so you need to be a good problem solver and be able to communicate your challenges and solutions to other team members.

Finally, I wouldn’t underestimate the benefits of being able to step back, see the bigger picture and generally have a sense of humour when developing software. For all the recent advances in “deep learning”, computers are ruthlessly unintelligent and will often frustrate even the most tenacious of developers. A fresh perspective, or a laugh, is sometimes surprisingly effective in untangling those hard problems.



AudioTelligence’s Aiso™ for Hearing Software Wins Innovator Award

Read more

Developing commercial software for untangling sound - 5 questions with Simon Harvey

Read more