Rogers Jeffrey Leo John, CTO of DataChat, shares insights on GenAI trends, agentic AI, SLMs, and building responsible, real-world AI systems.
Welcome, Rogers. To start, can you share a bit about your professional journey and what led you to co-found DataChat?
I’m Rogers Jeffrey Leo John, co-founder and CTO of DataChat, a natural language based no-code data analytics platform for business users. I completed my Masters in Computer Science at Columbia University, New York and Ph.D. in Computer Science at UW-Madison.
The quintessential problem DataChat solves is empowering business users to explore data and make data-driven decisions without needing to learn programming. As I was starting my Ph.D. back in 2015, data science was taking off in popularity and mainly businesses were starting to make data-driven business decisions. However, business users lacked the coding expertise required to interact with data systems and had to rely on engineers like me to get answers to their business questions so that they could make sound business decisions. More often than not, the engineers lacked the business acumen that would have helped them understand the question from the business user and provide data or answers in a meaningful way. And this resulted in a lot of inconsistent and improper business decision making. As a researcher, I could see parallels between research work that was happening in the Natural Language (NL) to SQL translation space and the inconsistent answer problem that the business users were facing. During my Ph.D. research, I focused on developing techniques to translate natural language into a wider range of data analytics workflows—including machine learning and visualization—beyond just NL to SQL. This work ultimately became the foundation of DataChat.
With discussions around AI’s evolution, do you think GenAI is reaching its peak in usefulness and accuracy? Where do you see it still providing value, and where is its impact diminishing?
GenAI continues to evolve, delivering value in areas where creativity and automation streamline human workflows, such as marketing, prototyping, and automation. It boosts efficiency in content creation, coding, and business decision-making while acting as a creative co-pilot across industries. Additionally, GenAI enhances personalization, offering tailored experiences in fields like customer service.
Gen AI in general is a great productivity multiplier – it allows you to offload repetitive and mundane tasks thereby freeing up your time for more impactful work.
However, GenAI’s impact is diminishing in areas where reliability and accuracy are critical. General-purpose chatbots face challenges with hallucinations and long-term context retention, making them less useful for high-stakes applications. The gap between AI hype and practical use is becoming more evident with skepticism in certain areas. Many projects in the industry are failing at the Proof of Concept (PoC) stage due to overexpectations of what AI can do and not delivering on its promises. Additionally, the high costs of running advanced models are pushing businesses to reassess whether the incremental improvements to existing tools and processes justify the investment. The future of GenAI lies in augmenting human expertise rather than replacing it.
The recent emergence of DeepSeek has sparked both excitement and concern. What do you see as its immediate impact, and what key questions does it raise the AI landscape?
DeepSeek definitely caused a stir when they released an open source model that was smaller in size and outperformed open AI’s models on several tasks. DeepSeek adopted a unique strategy, prioritizing efficiency over reliance on costly high-end chips. Their approach demonstrated that powerful AI can be achieved through intelligent software and hardware optimization and not just by heavily relying on supervised fine tuning. This efficiency-first approach disrupted assumptions that advanced AI requires billion-dollar investments.
DeepSeek’s model is completely open source. Anyone can finetune and deploy the model without any restrictions. This approach democratizes AI innovation by enabling startups, researchers, and developers to access advanced AI without licensing costs. It challenges the dominance of large AI firms, providing a strong alternative to proprietary models.
However, it also presents significant risks. Unlike proprietary AI, where companies can monitor and limit harmful uses, DeepSeek’s model can be repurposed by anyone, including malicious actors. This raises concerns around misinformation, and AI-driven fraud. Without proper safeguards, open AI systems could be exploited by malicious actors.
The complete open source nature of Deepseek has reignited the debate about AI policy and regulation. Should there be restrictions even on open source AI to prevent misuse? There are arguments on both sides of the table. Some welcome DeepSeek’s release as a victory for open science, which could spur innovation while others caution that unregulated AI could bring new risks that are beyond control.
The challenge of balancing AI accessibility with responsible management will define the future of AI policy and innovation. As technology advances, ensuring broad access while preventing misuse remains a critical debate. Striking the right balance will determine how AI evolves and impacts society
Agentic AI frameworks are gaining attention. Can you explain what they are and how do they differ from traditional AI models? What are the key advantages and challenges of adopting these frameworks?
Agentic AI frameworks are AI architectures designed for autonomous decision-making, planning, and adaptability. Unlike traditional AI models that react to direct inputs with predefined outputs, agentic AI systems proactively set goals, plan multi-step processes, and adjust their actions based on environmental feedback. This allows them to operate more independently and dynamically, handling complex tasks that require reasoning and iteration rather than just pattern recognition.
The key advantages of agentic AI include automation of complex workflows, improved adaptability to changing scenarios, and enhanced efficiency in problem-solving. These systems can integrate with external tools, retain memory across interactions, and make intelligent decisions, reducing human intervention. However, their autonomy also introduces challenges, including potential inaccuracies, difficulty in debugging, ethical concerns around decision-making, and increased computational resource requirements.
Despite these challenges, agentic AI frameworks are gaining traction due to their potential to transform automation and intelligent systems. Frameworks like LangChain, Auto-GPT, and OpenAI’s tool-use capabilities are enabling AI agents to perform increasingly sophisticated tasks. As these technologies evolve, addressing issues related to reliability, security, and efficiency will be critical to their widespread adoption in real-world applications.
We’re seeing multiple AI frameworks being developed, such as Microsoft’s Magnetic One and OpenAI’s offerings. How do these differ from each other, and do you think one approach will dominate the industry?
Both Microsoft’s Magnetic One and OpenAI’s Swarm are multi-agent frameworks that coordinate several autonomous agents, working together to accomplish complex tasks that may be too challenging for a single agent to manage.
The main difference between Magentic One and Swarm lies in how the agents communicate and coordinate with each other to accomplish a task. In Magentic One, there’s a centralized coordinator agent that coordinates with other specialized agents to accomplish a task. The coordinator keeps track of task status and assigns tasks to agents accordingly. In contrast to Magentic One, Swarm follows a decentralized architecture. Instead of relying on a centralized coordinator, the agents hand off to the next agent after they complete their task.
Both approaches have their own advantages and disadvantages. Magentic one is more structured, whereas swarm is more adaptive. The centralized coordinator can become a bottleneck for Magentic whereas agent-to-agent hand offs can be unreliable in Swarm.
I don’t think there’s going to be one approach that dominates the industry. There will always be applications where one approach is more suitable than the other.
The most widely adopted framework will be the one that emphasizes ease of development, seamless integration, and support for multiple LLMs.
Small language models (SLMs) are emerging as an alternative to both agentic AI and traditional LLMs. What are the primary use cases for SLMs, and which industries are leading their adoption?
Small language models (SLMs) are becoming increasingly popular as cost-effective and efficient alternatives to large language models (LLMs) and agentic AI. They provide benefits in terms of privacy, speed, and resource efficiency. Their primary applications include AI-powered assistants for customer service and automation, enterprise-specific AI for proprietary data processing, and on-device AI for mobile and IoT applications. Furthermore, SLMs are gaining popularity in the areas of code generation, personalized recommendations, security monitoring, and healthcare AI, where domain-specific optimizations enhance reliability and accuracy. They are well-suited for industries that prioritize data privacy, low-latency processing, and cost-effective deployment due to their ability to operate with lower computational requirements.
SLMs offer a practical alternative to resource-intensive AI systems by balancing performance and efficiency. This enables businesses to deploy AI solutions that are customized to their specific requirements without the high costs and energy consumption of full-scale LLMs.
Alibaba’s small AI model reportedly outperforms Claude while running on a standard laptop. Do you think SLMs will reshape AI deployment strategies, and what does this mean for accessibility and efficiency?
Small language models (SLMs) are demonstrating that bigger isn’t always better for AI tasks. For highly specialized applications, fine-tuning an SLM can unlock impressive capabilities without relying on large-scale models. Their efficiency makes them especially valuable in resource-constrained environments with strict data privacy needs—Apple’s on-device foundational models serve as a prime example.
SLMs have the potential to transform AI deployment by making it more accessible and cost-effective. Unlike large models that depend on powerful cloud infrastructure, SLMs can run efficiently on consumer grade hardware, cutting costs, reducing latency, and minimizing reliance on cloud providers. This approach also boosts privacy and security by keeping data on local devices, making AI more feasible for businesses and individuals with limited resources.
Further, SLMs enable enterprises to deploy multiple specialized models through a routing layer, allowing dynamic task allocation. This approach balances performance and infrastructure costs while maintaining model-agnostic flexibility. For example, separate SLMs can handle specific tasks without running parallel heavy models.
By enabling on-device intelligence in smartphones, IoT devices, and enterprise tools, SLMs can speed up AI adoption across various industries. Their decentralized nature removes barriers to innovation, allowing more companies and developers to experiment without the high computational costs. Ultimately, SLMs could pave the way for a future where AI is more accessible, affordable, and sustainable, transforming how businesses and consumers engage with AI-driven solutions.
BTW, fun fact: Generative AI advancements have been so swift that models now considered SLMs were seen as large language models (LLMs) just a year ago. Notably, many modern LLMs use a Mixture of Experts (MoE) architecture, where smaller, specialized language models handle specific tasks or domains. In this way, SLMs often play a critical role behind the scenes in enabling the capabilities of LLMs.
Looking ahead, what do you think AI architecture will look like in a year? Will we ever reach a point where models can learn continuously in real time, and what are the biggest obstacles preventing that?
Until recently the focus on Gen AI was on scaling during training i.e., improving model capabilities meant training larger models using more GPUs on larger datasets. More recently there has been a shift towards scaling test-time or inference time compute. This approach suggests that increasing compute resources when the model is generating responses can improve the accuracy.
Currently, the capabilities of such reasoning models are limited by inference infrastructure. The ability of these models to reason or think deeply increases memory and compute requirements. Thus model providers are forced to limit the reasoning steps to maintain reasonable latency and costs.
As more advanced inference systems are developed for next-generation AI workloads, reasoning models will gain the ability to dynamically adapt their reasoning depth based on available compute. This adaptability unlocks new possibilities for inference-time scaling, allowing AI systems to reach unprecedented levels of accuracy and robustness.
One of the main bottlenecks in adapting Gen AI for enterprise tasks is the limitation of data for fine tuning AI models. High quality human labeled data for training models can be difficult and expensive to obtain. So techniques that can leverage inference time scaling to address the data bottleneck, perhaps by automatically learning as users interact with the model, could be the first step towards continuous learning models.
Hallucinations remain a major challenge in AI. Do you see agentic AI as a solution, and if so, how? Can grounding data in databases or human collaboration help mitigate this issue?
Yes, one of the main challenges with Gen AI models is hallucinations. It’s important to provide the right context to these models to mitigate hallucinations. In that aspect, grounding LLMs in knowledge bases will definitely help the LLM by providing it with the right context.
However, completely eliminating hallucinations is an ongoing challenge. So it’s important for AI systems to leverage human expertise whenever it is appropriate. AI agents can help with mitigating hallucinations by shortening the loop of human involvement. Agentic AI can help mitigate hallucinations by iteratively verifying facts, retrieving data dynamically, and self-correcting based on conflicting information. Agentic AI can help mitigate hallucinations by iteratively verifying facts, retrieving data dynamically, and self-correcting based on conflicting information. By integrating retrieval-augmented generation (RAG) with active fact-checking, AI can cross-check sources, access structured databases, and refine responses rather than relying solely on probabilistic text generation. The Agentic AI can also decide when to involve the human in the loop.
By combining agentic AI with reliable external data sources and human feedback loops, we can significantly improve AI reliability.
On a personal level, what strategies do you use to stay ahead in the rapidly evolving AI space?
Staying on top of what’s happening right now in this space can feel like drinking from a firehose. Here are some things I do to try to keep up with what’s happening.
- Read Research Papers: Follow platforms like arXiv, Papers With Code, and Google Scholar for the latest AI papers. Two minute papers channel on Youtube contains snackable video summaries of papers from top AI conferences.
- Subscribe to AI Newsletters for curated updates.The Batch (DeepLearning.AI), The Gradient , AlphaSignal, Neuron are some of the examples.
- Follow Major AI Conferences. Track papers from NeurIPS, ICML, ICLR, CVPR, and ACL—leading conferences in AI and machine learning offer insight into the state of the art in AI.
- Read AI Research Blogs: Stay updated with blogs from Google AI, Nvidia OpenAI, DeepMind, and Meta AI.
- Watch AI Lectures & Talks: Follow Stanford CS, MIT CSAIL, DeepMind AI on YouTube for expert lectures and conference talks.
- More importantly, nothing beats hands on experimentation and exploration. There are open source tools and frameworks that you can leverage to experiment with and learn.
It’s unrealistic to keep up with everything consistently, so focus on a few reliable sources to avoid information overload. Maintain an effective balance between reading, engaging with the AI community, and hands-on experimentation to stay informed and ahead in the field.
Finally, what advice would you give to professionals and businesses looking to navigate the AI landscape effectively?
For businesses, 1) AI is not going to solve all problems magically. You need to have the right data, proper governance structures and processes in place to successfully incorporate AI in your organizations. Garbage in garbage out applies for Gen AI as well
2) Data Privacy – Be cognizant of what information you expose to the AI model. AI is great at learning quickly, but there’s no proven technique to make AI forget what it knows about you or your data.
3) Human in the loop: Do not over automate using AI. Have AI augment the human expert in making critical decisions. Trust, but verify!
4) Context is king. AI is good at many things, but there’s context that’s going to be unique to your business. Providing the right context to AI is crucial for successfully leveraging AI in your organization.
More importantly, think of AI as an accelerator and not as a shortcut. Invest in getting your data foundations right before you invest in AI.
Lastly, there’s a lot of hype around AI’s capabilities. While a simple proof-of-concept showcasing AI functionality may look impressive, transforming it into an enterprise-grade solution and driving adoption is an entirely different challenge. Always ensure that your AI project addresses a real business problem—otherwise, it’s just cool technology with no real impact.
Rogers Jeffrey Leo John
co-founder and CTO of DataChat
Rogers Jeffrey Leo John Ph.D., is the co-founder and CTO of DataChat Inc., a natural language based no-code data analytics platform for business users he helped establish in 2018. Rogers has held research positions at both the University of Wisconsin-Madison and Columbia University and has a strong background in Machine Learning, Natural Language Processing and Data Science.