profiq Video: Introducing Our Tech Research Project—The Autonomous Web Explorer Powered by GPT-4o

One of our Tech Research team’s most interesting recent projects is the development of an autonomous web explorer. This tool, built on OpenAI’s GPT-4o model, extends the capabilities of traditional web scrapers by performing more complex interactions, such as clicking buttons and filling out forms. This means it can navigate through dynamic states on websites—like encountering a login form with an error message about an incorrect password—by understanding and interacting with the content in context.

One of the primary uses of our automated web explorer is in the creation of knowledge bases for designing and automating test cases. By generating detailed models of how a website operates, developers and testers can design more accurate and comprehensive tests. This capability not only enhances the efficiency of test case creation but also ensures that the tests cover a wider range of scenarios.

Tutorial: Developing an AI Web Explorer

In this profiq tutorial, Miloš Švaňa walks users through the steps we took in developing our AI web explorer. Miloš is a PhD in Systems Engineering, ML engineer, and a profiq researcher specializing in large language models, deep learning, classical ML, and statistics.

Here are some useful time stamps in the video:
0-0:35 Intro
0:35 Goal of the Web Explorer
01:17 Why build an AI explorer instead of using a simple web scraper
01:50 Starting the explorer
02:56 Our Github address for this project

Access the tutorial here:

Follow Our Project on GitHub

You can follow the development of the autonomous web explorer and access the project repository here: profiq GitHub Repository. We welcome your feedback, suggestions, and any input you want to share. Be sure to keep an eye on our repository to stay updated on our latest advancements and contributions to AI tech!

You May Also Like:

profiq Blog: MLflow: serving LLMs and prompt engineering by Miloš Švaňa
profiq Blog: Evaluating LLMs with MLflow by Miloš Švaňa
profiq Video: How to download and run a LLM locally with LM Studio by Miloš Švaňa
profiq Video: Tech demo – Designing REST APIs with Stoplight by Miloš Švaňa
OpenAI: GPT 4o
YouTube Video: Vision-based Web Scraping with the New GPT-4o model by AI FOR DEVS