Senior Research Engineer- Optimization
Luma AI
Palo Alto, CA, USA
Posted on Sunday, May 26, 2024
Luma’s mission is to build multimodal AI to expand human imagination and capabilities. We believe that multimodality is critical for intelligence. To go beyond language models and build more aware, capable and useful systems, the next step function change will come from vision. So, we are working on training and scaling up multimodal foundation models for systems that can see and understand, show and explain, and eventually interact with our world to effect change.
We are looking for engineers with significant problem solving experience in PyTorch, CUDA and distributed systems. You will work with Research Scientists to build & train cutting edge foundation models on thousands of GPUs.
Startup Mindset
- Value velocity and execution.
- Communicate clearly.
- Focus on building what will matter to users and the product.
- Be resourceful at finding creative ways to overcome challenges.
Experience
- Experience training large models using Python & Pytorch, including practical experience working with the entire development pipeline from data processing, preparation & data loading to training and inference.
- Experience with profiling CPU & GPU code in PyTorch, including Nvidia Nsight or similar.
- Experience writing & improving highly parallel & distributed PyTorch code, with familiarity in DDP, FSDP, Tensor Parallel, etc.
- Experience writing high-performance parallel C++. Bonus if done within an ML context with PyTorch, like for data loading, data processing, inference code.
- Experience with high-performance CUDA and writing custom PyTorch kernels. Top candidates will be able to utilize tensor cores; optimize performance with CUDA memory and other similar skills.
- Good to have experience working with Deep learning concepts such as Transformers & Multimodal Generative models such as Diffusion Models and GANs.
- Good to have experience building inference / demo prototype code (incl. Gradio, Docker etc.)
- Please note this role is not meant for recent grads.
Benefits
- Equity grant to reflect the incredible value you will bring to Luma, with annual refreshes.
- Excellent salary and benefits.
- Full health, dental, and vision coverage.
- Latest and greatest gear.
- Stipends towards wellness, house cleaner, and phone/internet.
- Unlimited paid time off with 12 days minimum.
- Unlimited sick days.
Why Join Luma AI?
- You will get to work with the world best AI researchers, shipping their research to millions of users around the world
- You will be equipped with all the tools, technologies, resources and AI tools you need to get the job done
- We build. We ship. Your work will matter to people.
- We are building a very widely usable product, and you’ll get to work on equally wide variety of challenging problems.
- We have fantastic traction from early customers, whom you’ll get to work directly with.
- We have backing of some of the best VCs in Silicon Valley, and Angels from across the industry.
Your applications are reviewed by real people.