Using Generative AI for Concept Validation

I wanted to take a moment to share how I’m getting value out of generative AI tools like ChatGPT, Bing Chat, and GitHub Copilot.

This isn’t about hype. This isn’t about sample prompts. But rather, going into some detail around a specific circumstance where I’m finding these tools are providing value to me.

I’m calling this category “concept validation”, which can be used in the context of different tools or directly with Large Language Model (LLM) APIs. I find that this usually falls into a few more specific use cases:

1. There’s something I’m tangentially aware of, and want to validate that I’m on track before I dig in further
2. There’s something I’m familiar with, but want to discover related things that are otherwise not on my radar
3. I’m developing an application and I want to make sure I’m pursuing a reasonable approach

I like these because I have been able to get good feedback without a lot of time-sink into complex prompt engineering. There’s a place for that, but speed wins over perfection in this category.

Below I’ll elaborate a bit more on each.

Validating that I’m on the right general track

The idea here is to get a quick read on whether an idea or concept holds water, before proceeding forward.

Hallucinations aren’t the end of the world here. In fact, the possibility of hallucinations lets me feel more bold in challenging the response if I think I really am on the right track. But, importantly, I make a point to omit what I think the right answer is. I do this to make sure I don’t lead the AI to confirm my answer. Still, I prefer to frame in a way that minimizes the opportunity for bad responses.

Example: Let’s say that I think a motorcade can be used not just for political figures but also for emergency situations. I’ll ask “what circumstances warrant a motorcade”, instead of “can a motorcade be used for emergency situations”. The latter phrasing seems to be more prone to false positives than the former.

In this category, depending on the topic, I’m using either ChatGPT or Bing Chat.

Discovering what’s related

A common time I do this is when I have some specific knowledge, but want to fill in some gaps, or more specifically, gain an appreciation for which gaps I may want to fill in.

Example: as I’ve been getting more hands-on with Python, I like to stretch myself a bit further than a single concept. The question I ask resembles something like “What are some ways that people use ____________?” then I’ll sponge up those ideas, perhaps investigating, or perhaps just filing it away for another time.

For this category, I’m using Bing Chat the most. A big part of that preference is the citations and suggested questions. It becomes a way to do some of that exploration once I know what’s related.

I especially like the Edge browser integration for this, since I can use chat to ask a question without having to go anywhere. Additionally, I keep the “Page Context” option enabled, which can help summarize long pages without me even having to ask a question.

This also gets me excited about the forthcoming Windows Copilot…

Describe how this code will execute

I dabble in several different languages, and I can generally find a means to an end. That said, I do like having someone poke holes in what I’ve put together.

For code generation, I’m using GitHub Copilot primarily, though there are times when I additionally use ChatGPT when I want some additional history/context instead of just the code. I’m really looking forward to when GitHub Copilot Chat is available to me, so I don’t need to do the additional context switch.

Conclusion

Moving forward, I encourage others to explore the possibilities offered by generative AI tools and share the situations and context where you’re getting value.

The pace of change and innovation isn’t slowing down, and in fact I suspect these techniques won’t need as much explanation or elaboration in the future. That said, I do think that these are foundations worth exploring, which become the elements we can play with when creating more valuable abstractions and experiences.