Hey folks, remember when I wrote about my first experience with Azure OpenAI back in March 2023? I ended that post with a list of ideas for how we could use GPT in CloudLabs. Well, it’s been over a year, and I wanted to share where we actually landed with that.
Spoiler: some of those ideas worked out great. Others? Not so much. That’s how it goes.
Why AI in a Lab Platform?
If you haven’t read my CloudLabs post, the quick version: CloudLabs provisions hands-on lab environments at scale for tech training. Thousands of environments every month. Someone signs up for a workshop, clicks a button, gets a fully configured Azure environment.
The thing is, giving someone an environment is only half the problem. They still need to figure out what to do in it, troubleshoot when things go wrong, and actually learn something. That’s where we started thinking about AI.
GPT-4o just came out last month, Copilot Studio got announced at Build 2024, and Azure OpenAI has matured a ton since I first played with it. The timing felt right to actually build something instead of just prototyping.
What We’ve Built So Far
I’ll walk you through four things we’ve been working on. They’re at different stages of readiness, and I’ll be upfront about that.
AI Lab Guidance
This is the one I’m most excited about. When a user is working through a lab, they sometimes get stuck. Maybe the instructions aren’t clear enough. Maybe they accidentally skipped a step. Maybe they just don’t understand a concept.
We built an AI assistant that sits inside the lab environment and knows the context of what the user is doing. It has access to the lab guide, the current step, and some metadata about the environment. A regular chatbot doesn’t know that the user is on Step 7 of a Kubernetes networking lab. Ours does. We pass the lab guide content and the user’s current position as context to GPT-4o through Azure OpenAI, and the responses are actually useful.
I tested it on one of our AKS labs last week. Deliberately skipped creating a namespace, then asked the assistant why my deployment was failing. It caught the missing namespace immediately and walked me through creating it. That was cool.
Is it perfect? No. It sometimes gives overly detailed answers when a simple "you missed step 3" would be enough. We’re still tuning the prompts. But the early feedback from beta testers has been really positive.
Intelligent Troubleshooting
This one came from a real pain point. When a lab deployment fails (and with thousands of deployments, some always fail), our support team used to manually dig through ARM deployment logs, figure out what went wrong, and either fix it or escalate.
Now we feed those deployment error logs through Azure OpenAI and get AI-generated diagnosis and fix suggestions. The model is surprisingly good at reading ARM template deployment errors and telling the difference between a quota issue, a naming conflict, a region capacity problem, and a permissions error.
We started testing this in April. Average time to resolve a failed deployment dropped from about 12 minutes to under 4 minutes when the AI diagnosis is correct. And it’s correct roughly 75-80% of the time. The other 20-25% still needs a human, and that’s fine. We’re not trying to replace our support team, just free them up for the hard problems.
Lab Content Generation
Remember when I said in my Azure OpenAI post that GPT-3.5 Turbo could "cut the first-draft time in half" for lab instructions? With GPT-4o, it’s gotten noticeably better.
We built a tool where content authors describe what a lab should teach, specify the Azure services involved, and the AI generates a first draft of step-by-step instructions. The author then edits, tests, and refines.
I want to be careful here because I’ve seen too many companies claim "AI writes all our content!" and the result is garbage. That’s not what we’re doing. The AI draft is a starting point. Our authors still test every step, fix inaccuracies, and make sure things actually work. GPT-4o still hallucinates Azure CLI commands sometimes, less than GPT-3.5 did, but it happens.
What the tool really saves is the blank-page problem. Instead of staring at an empty document trying to structure a 45-step lab on Azure Synapse Analytics, the author gets a reasonable draft to react to.
Smart Lab Recommendations
This one is the most experimental and honestly the one I’m least sure about.
The idea: when a user finishes a lab, suggest what they should do next based on what they completed and their skill level. Like Netflix recommendations but for tech training.
We have a working prototype using embeddings to match lab content and user history. The recommendations are… okay. Sometimes spot on. Sometimes they suggest a beginner lab to someone who just finished an advanced one. We haven’t cracked the skill-level inference part yet.
I’d rather be honest about where this is than oversell it. It’s a prototype and it needs more work.
The Azure OpenAI Bits
Everything runs through Azure OpenAI with private endpoints. No lab data leaves our Azure tenant, which matters for enterprise customers. GPT-4o powers the guidance and troubleshooting features. The recommendations prototype uses text-embedding-ada-002 for embeddings.
Token costs are real though. The troubleshooting feature alone costs about $200/month during beta. We built caching for common error patterns so we don’t hit GPT-4o with the same failed-deployment log a hundred times. That brought costs down by about 40%.
What I’ve Learned So Far
Prompt engineering is more important than I expected. The difference between a mediocre prompt and a well-crafted one isn’t 10% better, it’s the difference between useful and useless. We spent weeks on our system prompts for the lab guidance feature alone.
Framing matters more than accuracy. We label the assistant "AI-powered (beta)" and tell users to verify answers against the lab guide. They’re totally fine with occasional mistakes. When we tested without the beta label? The same mistakes got much harsher feedback.
And the hardest part isn’t the AI. It’s the plumbing. Getting the right context to the model, managing conversation history, handling edge cases where the lab guide is poorly structured, dealing with rate limits during peak usage. The GPT calls themselves are the easy part.
What’s Next
We’re rolling out lab guidance and troubleshooting to more customers over the summer. Content generation is already in use by our internal team. Recommendations need more work before anyone outside Spektra sees them.
I’m also watching Copilot Studio, which Microsoft announced at Build 2024. If it matures the way I think it will, it could give us a faster way to build conversational AI without as much custom code. Too early to say, but it’s on my radar.
If you’re using CloudLabs and want to try the AI features in beta, reach out to me or the team at cloudlabs.ai. And if you’re building AI into your own product, I’d love to compare notes.
Happy building, folks!
Amit
Assisted by AI during writing
