OpenAI announces real-time API, rapid training and vision fine-tuning on GPT-4o for developers

OpenAI hosted its annual DevDay conference in San Francisco on Tuesday and announced several new upgrades to the application programming interface (API) version of ChatGPT, which can be repurposed and fine-tuned to run other applications and software. Among them, the main introductions are real-time API, fast training and fine-tuning vision with GPT-4o. The company also facilitates the model distillation process for developers. OpenAI also announced the completion of a funding round and stated that it had raised $6.6 billion (roughly Rs. 55 thousand crores) during the event.

OpenAI announces new features for developers

In several blog posts, the AI ​​company highlighted new features and tools for developers. The first is a real-time API that will be available to paid subscribers of the ChatGPT API. This new capability offers a low-latency multimodal experience, enabling voice-to-speech conversations similar to ChatGPT’s advanced voice mode. Developers can also use the six preset voices that were previously added to the API.

Another new introduction is the ability to quickly train in the API. OpenAI is introducing this feature as a way for developers to save costs on frequently used queries. The company noted that developers tend to send the same input prompts over and over again when editing a codebase or having a multiple conversation with a chatbot. With a quick tutorial, I can now reuse recently used input queries at a discounted price. Processing for the same will also be faster. You can check the new prices here.

The GPT-4o can also be fine-tuned for vision-related tasks. Developers can tune a large language model (LLM) by training it on a fixed set of visual data and improving its output performance. According to the blog post, the performance of GPT-4o for vision tasks can be improved with only 100 images.

Finally, the company also facilitates the model distillation process for developers. Model distillation is the process of building smaller, fine-tuned AI models from a larger language model. Previously, the process was convoluted and required a multi-step approach. OpenAI now offers new tools such as Stored Completion (to easily generate distillation datasets), Evals (to run custom evaluations and measure performance), and Fine-Tuning (fine-tuning smaller models directly after running Eval).

Notably, all these features are currently available in beta and will be available to all developers using the paid version of the API at a later date. Furthermore, the company said it will take steps to further reduce the cost of input and output tokens.