UPDATES

Integrating ChatGPT with DuckDuckGo – DEV Neighborhood


Earlier this morning OpenAI turned off Bing integration in ChatGPT. A few hours later we launched DuckDuckGo help for our chatbots. You are welcome ๐Ÿ˜

You may strive it out by clicking our AI chatbot button within the backside/proper nook of this web page and write one thing comparable to follows; DuckDuckGo “Pixar”

The double quotes are necessary, as a result of we use these to deduce what you need to ship to DuckDuckGo, and what’s the immediate that we use to semantically search our database, and what we’re sending to ChatGPT to rework your question into a solution of some type. The recipe for looking out is as follows;

DuckDuckGo “xyz search question right here”

At the moment it can solely return the highest 5 outcomes, and the question in the direction of DuckDuckGo is cached on the server for five minutes. However this may be modified simply.



The way it works?

Once you create a coaching snippet to your semantic search database, you possibly can add Hyperlambda code as an built-in a part of your coaching snippet. This Hyperlambda code shouldn’t be part of your coaching snippet as you create embeddings for it. After we generate vectors for our coaching snippet utilizing text-embedding-ada-002 all the things OpeAI sees is the next;

DuckDuckGo

This outcomes is that should you begin any question in the direction of our VSS database with the phrase “DuckDuckGo”, this snippet will bubble to the highest. This coaching snippet seems as follows;

{{
.url
strings.cut up:x:@.arguments/*/immediate
   .:"""
set-value:x:@.url
   strings.concat
      .:"https://html.duckduckgo.com/html/?q="
      strings.url-encode:x:@strings.cut up/1
unwrap:x:+/*
sign:magic.http.html2markdown-links
   url:x:@.url
   class:result__a
   max:int:5
   question:uddg
.end result
set-value:x:@.end result
   strings.concat
      .:"Beneath are the highest 5 hyperlinks I might discover at DuckDuckGo for the question ""
      get-value:x:@strings.cut up/1
      .:"":"
      .:"rn"
      .:"rn"
      get-value:x:@sign
return:x:@.end result
}}
Enter fullscreen mode

Exit fullscreen mode

Discover, we might write any static textual content earlier than and after the {{}} components. This textual content can be used throughout vectorisation – Nevertheless, for the DuckDuckGo use case, the above is adequate. As soon as the above snippet is getting used as context for ChatGPT, it can invoke [strings.mixin] for the coaching snippet. To see the end result you possibly can strive executing the next code in your Hyperlambda Playground assuming you’ve got created the above traning snippet to your foo mannequin and vectorised it.

sign:magic.ai.get-context
   kind:foo
   vector_model:text-embedding-ada-002
   immediate:@"DuckDuckGo ""Invoice Gates"""
   threshold:float:0.6
   max_tokens:int:2000
Enter fullscreen mode

Exit fullscreen mode

It will discover your DuckDuckGo coaching snippet, execute its Hyperlambda code, leading to one thing comparable to the next being returned:

DuckDuckGo

Beneath are the highest 5 hyperlinks I might discover at DuckDuckGo for the question "Invoice Gates":

* [Bill Gates | Biography, Microsoft, & Facts | Britannica](https://www.britannica.com/biography/Invoice-Gates)
* [Bill Gates - Wikipedia](https://en.wikipedia.org/wiki/Bill_Gates)
* [Bill Gates - Forbes](https://www.forbes.com/profile/bill-gates/)
* [Bill Gates meets privately with Xi Jinping as US-China tensions rise ...](https://www.cnn.com/2023/06/16/enterprise/bill-gates-china-xi-jinping-visit-intl-hnk/index.html)
* [Xi Jinping meets Bill Gates in China, calls him 'an old friend'](https://www.reuters.com/world/china/chinas-president-xi-meet-with-bill-gates-beijing-state-media-2023-06-16/)
Enter fullscreen mode

Exit fullscreen mode

That is then used as a “primer message” for ChatGPT or as “context” to reply the query. And the query after all is;

DuckDuckGo “Invoice Gates”



Utilizing ANY web site as an API

Our web site scraping expertise can (virtually) use any web site as an “API”. You may see this from the arguments handed into the above [magic.http.html2markdown-links] Hyperlambda slot. The above arguments are as follows;

  • url – URL to fetch. This one is dynamically constructed for the above DuckDuckGo components to retrieve HTML from DuckDuckGo with search outcomes matching your question.
  • class – This works as a filter, and can solely return hyperlinks with a CSS class title matching the required class. This lets you filter out any irrelevant navigation hyperlinks for our DuckDuckGo instance.
  • max – That is the utmost variety of hyperlinks to return.
  • question – That is in case the URL you are keen on is obfuscated inside some type of redirect URL, which is the case for DuckDuckGo. DuckDuckGo do not expose the precise URL, however fairly a “redirect URL”, which we’re capable of intelligently parse from the QUERY parameters to the unique URL, and return straight. For bizarre causes ChatGPT refuses to return URLs in the direction of DuckDuckGo, in all probability as a result of Microsoft paid them I assume …? ๐Ÿ˜œ

The above slot after all, turns any web site into Markdown, which ChatGPT occurs to completely perceive, permitting you to combine ChatGPT with the search outcomes returned from DuckDuckGo. The tip end result turns into as follows.

DuckDuckGo and ChatGPT

Discover, we do not really retrieve the web sites DuckDuckGo returns, regardless that we might in idea have accomplished this too, and used it as part of our dynamic context to reply actual time questions. Nevertheless, the power to have ChatGPT work with stay information implies we are able to have ChatGPT work with something – Together with your individual native database if you want. One thing we exhibit under.



Combine ANYTHING with ChatGPT

All in all, this enables us to actually combine EVERYTHING with ChatGPT, so long as you possibly can present us with the information we’d like. And we are able to use (virtually) any web site URL as an “API” for learn solely information, injecting this into the textual content stream we’re sending to ChatGPT, having ChatGPT work on actual time information.

You are welcome ๐Ÿ˜

FYI: All the safety considerations OpenAI used as arguments to close off Bing help doesn’t apply for our answer ๐Ÿ˜Ž

Leave a Reply

Your email address will not be published. Required fields are marked *