Stay Humble, Stay Curious and Stay Positive : Some Thoughts from Research vs Engineering

A blog talks about difference between research and engineering, and my takeaway from it.

Recently, I read an article from Lecun about difference between research and engineering. There was one takeaway I hope I could remember for my whole life: stay humble, stay curious, and stay positive.

As a young person transitioning from a heavy-industry environment at Uwaterloo to someone pursuing AI theory at least now. I have some thinking about difference between research and engineering.

Table of Contents

• My Understanding From LeCun’s Distinction
• My Understanding From Musk’s View
• A Lesson From My Friend Matt
• My Life Philosophy
  • Stage One: Idealizing Industry
  • Stage Two: Realizing the Gap in Engineering
  • Stage Three: Recognizing Both Worlds
• Conclusion

My Understanding from Lecun’s Distinction

Lecun describes research as a proufoundly creative work to increase the stock of human knowledge. Researchers care about explianability, elegant principles and work that can be critized, reproduced by the research community.

Engineering, in constract, integrates developed research discovery into working systems. The philosophy of engineering is pragmatic: choose the first sets of scientific tools that work well given real-world constraints. Whether the method is the abosolute best matters less than wheather it is good enough to work.

Metrics also differ. Research often uses offline metrics liek precision, recall etc. Engineering adds online metrics which is problem-oriented such as click-through rate, revenue lift etc.

My Understanding from Musk’s View

Musk said that, at xAI, ‘there are only engineers’. I think it conflicts with Lecun literally but there is similarities behind the words.

To Musk, engineers are driven by their small steps - KPI which will eventually lead to something transformative whether generating revenue or some great ambition like exploring the extention of humananity. Musk as a great enterpreneur is a steadfast builders. His management philosophy is gounded in efficinecy and reality: the value you create determines what you get paid.

Musk’s idea is very programatic and problem-solving oritented. I would call it engineering thinking resonating with Lecun’s engineering idea.

TL;DR

On engineering side, the work of engineers forms the link between scientific discoveries and their applications to human life and businees. The metric for engineers are their product impact often related to the companies’ ambition.

On reserach side, the goal is fundamentally different. Research is creative pursuit focused on truth and correctness. The metric for researchers are their intellectual impact.

A Lesson form my Friend Matt

There is some important I learned from my best friend Matt at UWaterloo who is working at X now. You have the right to choose the company you want to work.

If you don’t like a company’s philosophy or rules, you can choose one aligning better with your life values. Musk’s companies have high workload, gender imbalance, and research-engineering tension - but many other companies are more inclusive, better wlb, or more reserach-friendsly.

Just goes with the one fits you.

My life Philosophy

First, I want to admit my ignorance. Everyone carries their own bias and all of us are constained by the perspectives we start with. Thanks for my girlfriend Zhiqi to point out my ignorance. My journey so far can be divided into three stages, each revealing something different about myself and my major knowledge level.

First Stage: Idealizing Industry

In the beginning, I believed industry was far better than academia.

When I decided to explore research at Waterloo, it was because I felt frustrated by the “probability game” behind modern AI. I wondered:

Why must artificial intelligence rely so heavily on probabilistic models? Is there a way for AI to access knowledge directly, rather than follow the probability game?

At that time, I am really weak at math, for now, I am still weak at math. I am confused why people should learn these really hard math to do AI. I thought maybe IR(information retrieval) and RAG(Retrival-Augmented Generation) could be a way to free us from the probability game. But after learning the details, I realized that RAG simply intergrates the traditional IR pipeline to get the top k documents and put these documents and prompt into LLM to generate the answer. It’s still within the range of probaility game. I know many researchers on theory side would be very mad after hearing my idea:)

It’s a failure but it’s also an important moment of remider which says don’t judge.

Now, I want to study math precisely even I am dense to math, not reject them blindly. My goal is to get good command of math tools so that I can rigorously verify my ideas. And I believe these math tools has been validated through decades of research. And these theory remains gold, even now in a fast-moving AI field.

Second Stage: Realizing the Gap in Engineering

After a couples months of learning in Ottawa, I gradually udnerstnad that many engnineers(previous me is part of them) don’t know the principles behind the tool. They move fast, build things, and deliver results, but sometimes without understanding the deeper structure.

During my time in Ottawa, Professor Mao taught me something I value deeply:

• how to formulate a question
• improve my mathematical thinking ability

For the first time, I felt like I was on the right track - not just building things, but understand why they work.

Third Stage: Recognizing Both Worlds

Recently, I was running and playing around with some LLM structure which is on engineering side; I realize that instead of treating the two things seperatedly. We could think these as two skills: think in a clean enviroment and implement in a dirty enviroment.

So I think the most important things for me is What of Probelm I Want to Solve? Then I choose What Kind of the Skills I Want to be Good at.

Conclusion

All in all, the real question is only one:

Which Problems I want to solve?

I still don’t have the answer for this question, but I think that’s okay. It need time to get it. What matter is I am learning to stay humble, stay curious, and stay positive as I navigate during my journey.