<p>Hi there! My name is Oscar. I’m a summer intern at LeapMind and a Taiwanese student from Waseda University. I had an incredible time interning at this rapidly growing, multinational startup and would love to share my journey with you!</p>
<h2 id="making-deep-learning-prevalent">Making Deep Learning Prevalent</h2>
<p>This original title from LeapMind caught my eye during my internship hunting. With IoT becoming more common, it is refreshing to see this new concept of bringing deep learning to various devices. After learning about the company’s mission of embedded deep learning and its multilingual, open working environment, I knew that an internship at LeapMind is going to be unique for me. The experience of working at a Tokyo startup that designs hardware and software for deep learning is just an opportunity I cannot miss!</p>
<h2 id="internship-project-optical-flow-estimation-with-blueoil">Internship Project: Optical Flow Estimation with Blueoil</h2>
<p>The goal of my internship is developing a DNN for optical flow estimation with Blueoil and quantizing the network for edge devices to run. Blueoil is an open-source software from LeapMind that helps me apply quantize-aware training and convert a trained network into an executable program. By running this program on a CPU or FPGA-based device, embedded deep learning is achieved! 😎</p>
<p>I worked on this project with an intern partner, Inoue san, and my mentor, Iizuka san. My main task is designing the network for optical flow estimation using TensorFlow and verifying its performance after quantize-aware training. After surveying multiple related works and considering their implementation costs and compatibilities with Blueoil, we decided to implement and quantize FlowNetS, a famous DNN for learning optical flow, published in 2015.</p>
<p>This task might look simple at first glance. All I have to do is reproduce a DNN and apply quantize-aware training to it. In reality, there are numerous constraints and challenges:</p>
<p>In Blueoil, only a subset of TensorFlow operations in FlowNetS can be converted for edge devices, so I must replace many parts of the network with original architectures.
Blueoil uses extreme low-bit quantization (1-bit and 2-bit), which can significantly weaken the network's performance. Therefore, I need to consider different speed-accuracy trade-offs, e.g., which part of the network to quantize.
The dataset for optical flow estimation is large (30GB), and the total training time takes more than 2 weeks. It usually takes around 2 days to tell whether the network is learning effectively or not.</p>
<p><img src="/images/posts/en/news/deep-learning-internship-at-leapmind/4f216710-img_7065.jpg" alt="Image: Presentation"></p>
<h2 id="results--what-i-learned">Results &#x26; What I learned</h2>
<p>Despite all the difficulties mentioned above, I was able to overcome them thanks to the support and advice from Inoue san, Iizuka san, and multiple full-time members. In the end, I developed an original DNN, LmFlowNetS, and reduced its inference time by 53.14% on a CPU-based device with quantization! Inoue san and I even gave a live demo of this network during our final presentation. Through this experience, in addition to technical skills, I have also learned the following:</p>
<ul>
<li><strong>Take the initiative and don't hesitate to ask the experts.</strong></li>
<li><strong>Communicate and listen actively.</strong></li>
<li><strong>Observe carefully and experiment boldly.</strong></li>
</ul>
<p>To sum up, I had a rewarding experience working at LeapMind. The vibrant, open company culture really keeps me motivated every day. If you're interested in gaining an experience similar to mine, please don't hesitate to apply!</p>
<p>Our Source Code: [<a href="https://github.com/OscarWang114/blueoil/tree/lm-flownet-s">https://github.com/OscarWang114/blueoil/tree/lm-flownet-s</a>]
Final Presentation Slides: [<a href="https://www.slideshare.net/leapmind/final-presentation-optical-flow-estimation-with-dl-183441802">https://www.slideshare.net/leapmind/final-presentation-optical-flow-estimation-with-dl-183441802</a>]</p>
<p><img src="/images/posts/en/news/deep-learning-internship-at-leapmind/f405f06d-img_7117.jpg" alt="Image: Presentation"></p>
<h2 id="bonus-what-i-love-at-leapmind">Bonus: What I love at LeapMind</h2>
<p><strong>Open culture🙌</strong>: Everyone at LeapMind is very approachable and transparent. I don’t need to hesitate to ask anyone questions, even the CTO.
<strong>Shuffle lunch😋</strong>: This regular event divides everyone randomly into groups for lunch. It’s the perfect chance to socialize with people from other teams!
<strong>Technology💻</strong>: I love the free FPGA boards for development and the object detection live demo at the office entrance!</p>

Deep Learning Internship at LeapMind

<h2 id="leapminds-employee-interview--msl">LeapMind’s Employee Interview : Ms.L</h2>
<p>I started my career as a software engineer in Altera Corporation (now part of Intel Corporation) back in my home country, Malaysia. It was 2010 that I took a sharp turn in my life to further my study as a Doctoral candidate in Tokyo Institute of Technology. I spent some years to research about multiprocessing IO system, targeting Linux kernel architecture. With the opportunity of research project collaboration, I returned as an Embedded System Engineer in a company located in Tokyo, and it was then in 2018 Feb that I joined LeapMind as a Hardware Engineer.</p>
<h2 id="what-are-you-responsible-for-at-leapmind">What are you responsible for at LeapMind??</h2>
<p>As a Hardware Engineer, I am in charge of FPGA/ASIC development to accelerate our deep learning solution. Keeping a constant growth and having continuous skill sharpening are also parts of my responsibilities as an engineer in LeapMind. It is my mission to seek for constant breakthrough to achieve the best accelerator, together with my team members in LeapMind.</p>
<p><img src="/images/posts/news/employee-l-t/af2386e8-interview004-660x440.png" alt="Work in progress Ms.L." title="Work in progress Ms.L."></p>
<h2 id="what-made-you-decide-to-join-leapmind">What made you decide to join LeapMind?</h2>
<p>It was the potential of LeapMind to excel as a future AI outstanding company that attracted me to join LeapMind. LeapMind comes with a great vision and I was impressed by the strong faith and passion that hold by the CEO of LeapMind. On top of that, LeapMind is a company with multinational environment. LeapMind accepts talents from worldwide and aims to be an international company.</p>
<h2 id="what-is-important-to-you-in-your-job">What is important to you in your job?</h2>
<p>As an engineer, it is important for me to have creative and innovative idea to come out with fast and quality solution. Teamwork is a significant element to join the intelligence and forces of the team members in order to achieve the best outcome. It is also important to me to keep improving myself in term of technical skills and soft skills so that I could be a contributive member in the team.
Last but not least, in order to have the best physical and mind state in work, the flextime system in LeapMind is such a merit for me as I am an extreme early bird. With the minimum required core time, I can work more productively and leave office as early as 3.30pm. This is really important for me to keep a work-life balance healthy lifestyle, especially to avoid the stress caused by the train peak hour congestion.</p>
<h2 id="what-inspiration-do-you-get-from-the-members-around-you">What inspiration do you get from the members around you?</h2>
<p>There are always role models around me in LeapMind. There is a very particular team member of mine that has impressed me with his passion in his work, his professional soft skills, and his amazing engineering skills despite his young age. Above all, it is his positive attitude that makes him a valuable engineer in LeapMind. I also get inspiration from the leaders in LeapMind, as they dare to challenge the great vision. There are always something to learn about from each member in LeapMind, and I will continue treasuring for it.</p>
<h2 id="what-do-you-think-about-leapmind">What do you think about LeapMind?</h2>
<p>I've been enjoying working with LeapMind since I joined the company. We have nice and cool members in LeapMind, and I really enjoy working with my team members. There are always space to grow in LeapMind since we deal with the cutting-edge technologies and we are always free to try our new ideas. The working atmosphere is really great here as members are willing to help each others, to share their knowledge and skills, and also to share the happy moments together.</p>

LeapMind’s Employee Interview : Ms.L

<p>Before joining LeapMind in November 2017, I was studying at Warwick University in the U.K. First an MSc. in complexity science, then a PhD. in physics and complexity science. Making simulations to model physical processes was a big part of my work. This involved machine learning for optimization, probabilistic sampling methods, and some group theory mathematics.</p>
<h2 id="what-are-you-responsible-for-at-leapmind">What are you responsible for at LeapMind?</h2>
<p>In LeapMind, I am making use of my programming and scientific skills in the research and science team. My role includes generating new ideas for efficient deep learning technology, as well as making improvements to existing deep learning methods. I'm also invested in the question of how it is possible for neural networks to learn, and I believe more research in this area is important for making big improvements in this field.</p>
<h2 id="what-made-you-decide-to-join-leapmind">What made you decide to join LeapMind?</h2>
<p>When I saw the interesting new technologies occurring in deep learning, I felt it was an exciting field to take part in. I had some experience in creating new algorithms and making existing algorithms faster, so I felt like I could do some good work at LeapMind specifically. On the human side, the idea that people’s lives could become easier by intelligent devices really appealed to me.</p>
<h2 id="what-is-important-to-you-in-your-job">What is important to you in your job?</h2>
<p>To continue to grow, get new skills, and make a positive contribution are all important for me. Of course, I have interest in mathematics, machine learning, and logical investigation, so I am especially happy to be able to be involved in using these skills in my daily work.</p>
<p><img src="/images/posts/news/employee-j-n/image1.png" alt="Work in progress Mr.J." title="Work in progress Mr.J."></p>
<h2 id="what-inspiration-do-you-get-from-the-members-around-you">What inspiration do you get from the members around you?</h2>
<p>I am coming from a theoretical background. So for me, it was an experience to see the other side of technology, involving software engineers, where code is implemented into working products. Also, the hardware side, where the details of specific components are key. The spirit to create with others in the company is an inspiring feeling.</p>
<h2 id="what-do-you-think-about-leapmind-please-tell-us-what-you-like-about-working-at-leapmind">What do you think about LeapMind? Please tell us what you like about working at LeapMind.</h2>
<p>In LeapMind, questions are valued, so I don’t need to worry to ask. Also, collaboration across the company is valued highly. I can speak with people from all different sections, which I think does not always happen in other companies. There are people from various backgrounds, for example home country or work background, so I feel lucky to meet lots of different kinds of people. We have fun playing table tennis, and going to the pub outside of work, too. We recently held a hack-days event inside the company, and I wrote a command-line tool for inspecting network weights, it's a nice tool we can use for visualizing our daily work. One of the things I like about LeapMind is that there is a good culture of encouraging commitment to following through on project ideas to its conclusion.</p>

News