Even inexperienced drivers feel safe. Towards zero traffic accidents in society
~Development Trajectory of the In-vehicle Rear Camera with Pedestrian Detection Function~
While automobiles are becoming increasingly popular and convenient with enhanced functions to support driving, there is no end to the number of traffic accidents. The installation rate of in-vehicle rear cameras is increasing, as evidenced by the standardization of in-vehicle rearview cameras in North America (FMVSS 111), and the refinement of standards in Japan for devices for means of rear visibility or detection (UN-R158) in vehicles.
In this feature article, we introduce some of our employees who are hard at work on site to create a society with zero traffic accidents.
We interviewed Mr. Shirai of the Automotive Systems Business Division, who oversees the development of the in-vehicle rear camera system. Mr. Shirai always approaches work from the customers’ perspective, enjoying not only the process of developing products, but also the excitement of creating something new as he works on the task at hand. Seeing Mr. Shirai, the leader, enjoying himself, all the members also enjoy their own work.
First of all, please tell us about your career to date.
“My first assignment after joining the company was to develop a non-linear video editing equipment. Although it has become commonplace now, the equipment enabled users to edit videos by loading images into a computer, and I was allowed to go to the Nagano Olympics in the middle of development. Through this experience, I decided that I wanted to continue working on products that I develop, and to work directly with the customers in places where I can listen to their feedback and see their faces.
Just when I was beginning to settle in at my job, I was assigned to develop a TV master system for broadcasting stations to start preparing for the transition to terrestrial digital broadcasting in 2003. In this role, I experienced working both as a developer and as a system engineer (SE), deciding on specifications directly with the customers and negotiating prices. Although I now handle other products, I find it rewarding working together with the customers and everyone else involved to pursue the same goal.”
With the experience he has gained there, and at the invitation from his former boss, he is now with the Development Division for in-vehicle products. Please tell us about the subject of your current development.
“The second-generation pedestrian detection camera. The idea for the first generation started in 2013. I have been involved in the development of the first generation since 2014. Once completed, it would be the world's first product development back then, so we faced enormous obstacles during development and had a very hard time. We started this project as a joint development with a car manufacturer. Since then, we have been working on the project with the passionate belief that adding a pedestrian detection function to in-vehicle cameras will create new added value. I remember how excited we were about creating something that didn't exist in the world.”
What difficulties did you face in the development of the second-generation camera?
“When we started the development of the second generation in 2017, we worked on implementing improvements to handle the issues that we were unable to resolve when we commercialized the first generation. In the first generation, we had a lot of difficulties ensuring that the product met quality requirements before mass production. Once, we received comments that drivers would immediately complain if the alarm or the brake control is activated even when there are no pedestrians, so it was essential for us to address the issue before mass production. We had a hard time because we were racing against the clock.
However, whining does not solve anything. We tried to understand what exactly was going on and, in some cases, visited the actual site to determine our next course of action calmly. We have always made it a point to find solutions, try them out, and verify them quickly, and to think about how we can repeat this process over and over again quickly. By doing this continually, we were able to gain the trust and confidence of the car manufacturers and accumulate more and more know-how.
What I kept in mind and followed through until the end was that, as the person overseeing the project, I had to take the lead to face the customers and communicate directly with them. In general, as the development and design team, we have regular meetings with the car manufacturers to report on our progress, but some of us are not very good at dealing with customers. Rather than leaving it solely to the members in charge, I joined them in the discussions to encourage them to build up their confidence and experience. I want to personally show that issues occurring in the development process give us the opportunity to earn the trust of the car manufacturers. With this in mind, I actively visited the customers and went to customer sites where I could talk with them. I believe it is important for us to brainstorm good ideas with our customers, not as car manufacturers and suppliers in a hierarchical relationship, but as partners working together in a equal relationship to create cars that are appealing.”
What is the key technology behind the new in-vehicle rear camera?
“A major technical breakthrough is the OFF function. Since this is a pedestrian detection camera, it must of course provide the basic functionality of detecting pedestrians. However, we also took great pains to get the notifications right—we developed algorithms to ensure that the camera does not send out false notifications to drivers about pedestrians when there are no pedestrians, or dirty lenses when the lenses are clean. Together with the team members, we have been working through trial and error to find the best ways to apply new algorithms to solve the issues.”
The product finally took shape and I believe it was introduced to the market as the world's first technology at that time. How did you feel back then?
“To be honest, I had doubts when the product was completed (laughs). I know all the details behind the development process, so I couldn't help but see only the issues. Perhaps it is in the nature of developers and designers to feel this way. During the development process, we would receive a lot of comments from customers and from other members in the team, so we tended to lose confidence. But it was the first of its kind in the world back then. I want to create an atmosphere where the members involved in the project can hold their heads high and tell their families and acquaintances that they are working on such an amazing product. When young members in particular are motivated, they try out all kinds of interesting things without any prompting, so it is essential to motivate them.”
“I have digressed somewhat. The second-generation pedestrian detection camera is already installed in automobiles and shipping has already begun, but the truth is, I have not had the chance to ride in one installed with the camera because of the COVID-19 pandemic and because the camera is mainly installed in automobiles for markets outside Japan. Soon, it will be installed in vehicles for the Japanese market, and I look forward to experiencing it for myself when the time comes.”
How do you plan to expand the use of this pedestrian detection camera in the future?
“Specifically, we would like to focus on expanding sales in the Chinese market. In China, not only are people very receptive to the latest technologies, they also have a very high level of safety awareness. By deploying highly sophisticated pedestrian detection cameras in the Chinese market, we hope to create a safe and secure automotive society while keeping costs down.
Due in part to increased safety awareness, functions associated with automobile safety are attracting a lot of attention, so our major goal is to get more pedestrian detection cameras installed in more cars. Currently, the camera is installed in high-end models, but we would like to see it installed in mid-range models to enable many more people to ride in them. As an engineer, I hope to apply the many insights that I gained through this development to our next core products and play a role in creating appealing products that are uniquely Panasonic.
It would be great if we could make a difference—even if in some small way—in creating a society with zero traffic accidents by creating industry-leading Advanced Driver Assistance Systems (ADAS) like this one. However, we are still halfway there. I am always on the look out for hints in everyday life to handle the next challenge at work. Going forward, I would like to continue developing products that can address the feedback of end users who, like me, are excited about and look forward to riding in cars.”
Euro NCAP* trial using the pedestrian detection camera
*Euro NCAP: European New Car Assessment Programme, an automobile safety testing program conducted in Europe