Live Chat With Tomás Hernando Kofman about NotDiamond!

1. NotDiamond is an AI model router that optimizes query responses.

🥇92 17:40

NotDiamond automatically selects the best language model for a given query, enhancing accuracy while reducing costs and latency.

• It outperforms major foundation models on benchmarks.
• The system allows developers to customize routing based on their data.
• This approach addresses the complexity of managing multiple AI models.

2. The future of AI involves multiple specialized models.

🥈89 20:41

The trend is moving away from a single monolithic model to a network of specialized models for different tasks.

• Smaller models can be more efficient and cost-effective.
• Different models excel in various areas, such as logic or writing.
• This shift allows for greater computational efficiency and flexibility.

3. Open sourcing enhances the AI routing ecosystem.

🥇90 22:18

NotDiamond has open-sourced a model router to promote specialization and efficiency in AI applications.

• The open-source model outperforms existing options in terms of performance.
• It encourages innovation and reduces dependency on single providers.
• Open sourcing fosters a more competitive and diverse AI landscape.

4. Developers can customize routing based on specific needs.

🥈88 29:11

NotDiamond allows users to define their own routing preferences and evaluate model performance.

• Users can upload their evaluation data for tailored routing.
• The system learns from user feedback to improve recommendations.
• This customization leads to better performance for specific applications.

5. NotDiamond offers low-latency model routing for applications.

🥇92 29:28

The NotDiamond service provides efficient model routing with low latency, especially when deployed locally, achieving around 50 milliseconds for inference speed.

• API calls may introduce additional latency, typically around 400-500 milliseconds.
• The service can route queries to faster models, further improving response times.
• Local deployment allows for seamless integration into existing infrastructures.

6. Integrating NotDiamond into existing applications is straightforward.

🥇90 30:38

Developers can easily integrate NotDiamond by modifying just a couple of lines of code in their existing applications using OpenAI API.

• The integration involves specifying a list of models instead of a single string.
• NotDiamond acts as a recommendation layer before sending requests to the language models.
• This approach does not require significant changes to the existing application pipeline.

7. NotDiamond supports a wide range of models and rapid onboarding.

🥈88 32:41

Currently, NotDiamond supports around 40 models and can onboard new ones within 24 to 48 hours based on user requests.

• The onboarding process is efficient, allowing for quick integration of requested models.
• User feedback plays a crucial role in determining which models to prioritize for integration.
• The service aims to align model offerings with developer needs rather than just research trends.

8. Model performance varies based on specific use cases.

🥈85 35:01

Different models excel in various tasks, with user feedback helping to refine routing decisions for optimal performance.

• For example, certain models perform better for JavaScript tasks, while others excel in Python.
• User feedback is essential for improving routing accuracy and model recommendations.
• Not all models are suitable for every question, and using them judiciously can save costs.

9. Leveraging LMS for customer support and HR tasks is effective.

🥈88 45:27

Learning Management Systems (LMS) can significantly enhance customer support and HR tasks, although some complex queries still require human intervention.

• Certain tasks depend on implicit institutional knowledge that LMS cannot handle alone.
• Identifying when to route queries to humans is crucial for effective use.
• Training custom routers can improve performance in narrow domains.

10. Incremental improvements in LMS can yield substantial benefits.

🥇92 47:41

Even small performance gains in LMS can lead to significant quality improvements, especially for large-scale applications.

• Companies can achieve better outcomes by optimizing routing between models.
• Incremental gains can exponentially enhance the quality of outputs in recursive LMS usage.
• High-scale use cases make these improvements extremely valuable.

11. Cost reduction while improving quality is achievable with LMS.

🥇95 48:40

Users have reported significant quality improvements while simultaneously reducing costs, showcasing the efficiency of LMS.

• One user increased output quality by 72% while reducing costs by 14%.
• Another case showed a 30% cost reduction with slight accuracy improvements.
• This dual benefit is a key advantage of the LMS paradigm.

12. Routing models can optimize performance across multiple languages.

🥈87 56:03

The LMS supports multiple languages, allowing for optimized routing based on language-specific model performance.

• Routing is not just based on language but also on the context of the query.
• Different models may excel in various languages and contexts, enhancing overall effectiveness.
• This capability broadens the usability of LMS in diverse applications.

13. Custom routing can significantly enhance model performance.

🥇92 1:01:05

By training a custom router on specific workflows, users can improve output quality by 72% while reducing costs by 14%.

• This approach outperformed the previously best model, Sonet, in their specific domain.
• Routing between multiple models allows for nuanced handling of tasks like rewriting.
• The method emphasizes the importance of tailored solutions over generic models.

14. The name 'NotDiamond' reflects philosophical concepts.

🥈88 1:02:20

NotDiamond references modal logic, indicating the limits of expression and the evolution of communication technology.

• The term 'not possible' highlights the boundaries of what can be expressed.
• Philosopher Ludwig Wittgenstein's quote emphasizes the constraints of language.
• The name also has aesthetic appeal, contributing to its choice.

15. NotDiamond offers free queries to users.

🥈85 1:04:04

Users can access 100,000 free queries per month, with a minimal cost thereafter.

• This model encourages experimentation and usage of the platform.
• The pricing structure is designed to be accessible for users.
• Engagement through free access can foster a larger user base.