Optimizing AI Tools: The Power of Workflow Adjustments

Article Summary

Companies often introduce new tools to improve how they work. Copilot code review, an AI system that checks software code, decided to switch from its own specialized tools to a set of shared, better-maintained tools used by other Copilot products. The idea was that these shared tools, which included functions like grep and glob, would make the code review process more efficient and reduce costs. Initially, however, tests showed the opposite result: the cost of reviews went up, and the system found fewer issues. This was a surprising problem, as the new tools were considered superior.

The team soon realized that the tools themselves were not the problem. Instead, the instructions given to the AI agents for using these tools were not suitable for code review. The agents were acting like general assistants, exploring code broadly, rather than focusing on specific changes in a pull request, which is what a human reviewer does. To fix this, the team rewrote the instructions, guiding the agents to start from the specific code changes and then narrow their search for relevant information. This new approach led to a significant improvement. After adjusting the workflow instructions, the average review cost dropped by about 20% while maintaining the same quality. This success highlights that matching the tool's usage instructions to the specific task is crucial for achieving good results, even with advanced technology.

Key Vocabulary

Comprehension Questions

1. What was the initial expected outcome when Copilot code review switched to shared tools?

• A decrease in review quality.
• An increase in review cost.
• A more efficient review process and lower costs.
• No change in review performance.

2. What was identified as the main reason for the initial poor performance of the new tools?

• The new tools were technically inferior.
• The AI agents were not trained enough.
• The instructions for using the tools were unsuitable for the task.
• There was a lack of shared infrastructure.

3. What can be inferred about the way human code reviewers typically work, based on the article?

• They prefer to explore the entire code repository broadly.
• They focus directly on specific changes and seek minimal relevant information.
• They avoid using advanced search tools.
• They always need detailed instructions before starting a review.

4. Why did simply swapping the old tools for the new shared tools not work as expected?

• The new tools were too complex for the AI agents to learn quickly.
• The original tools were more powerful than the shared tools.
• The workflow instructions for the new tools did not match the specific needs of code review.
• The company did not invest enough in testing the new tools before implementation.

5. What is the most important lesson for businesses when integrating new AI tools, according to the article's outcome?

• Always choose the newest and most advanced tools available.
• Ensure the tools' instructions and workflow align with the specific job.
• Prioritize reducing costs above all else during implementation.
• Avoid using shared tools across different products.

Discussion Prompts

1. Can you share an experience where updating a tool or system in your workplace initially caused more problems than it solved? What was the underlying reason?

2. How important is clear, specific guidance for your team or colleagues when they adopt a new technology or change a process? Provide an example.

3. The article shows that adjusting 'workflow around the tools' was key. How do you ensure that new tools fit well into your company's existing workflows, or how do you adapt workflows to new tools?

Live Session Prep & Cheat Sheet

🎯 Speaking Targets (Vocabulary)

Try to use these target terms in your speaking turns:

• Pull Request
• Workflow
• Benchmark
• Efficient
• Effective

⚙️ Grammar Target Formula

Expressing Cause and Effect: Cause + (because of / due to / as a result of) + Effect

💬 Discussion Openers

Use these phrases to open or structure your arguments:

• In my experience...
• I think a key point here is...
• That reminds me of a situation where...
• To add to that...
• What are your thoughts on...?

Teacher Notes

This lesson focuses on a common business challenge: integrating new technology successfully. Emphasize that 'technology' isn't always the solution itself, but how we use it. Encourage students to relate the concepts of 'workflow' and 'instructions' to their own professional experiences. The grammar focus on cause and effect is vital for clear communication in business discussions and reports. Ensure students understand the subtle differences in usage for the various cause/effect phrases.

Speaking Class Facilitation Guide (Tutors/Moderators Only)

🎭 Role-Play Scenario

Situation: Your company recently implemented new customer relationship management (CRM) software. It was supposed to streamline sales processes, but the sales team reports that it's actually slowing them down and they are missing key customer details. You need to discuss the problem and decide on the next steps.

Goal: Identify the root cause of the problem (software, instructions, or workflow mismatch) and agree on a concrete action plan to improve the sales team's efficiency with the new CRM within the next month.

⚖️ Debate Prompt

{"side_a":["Adopting new technology requires companies to modernize. Adapting workflows ensures they fully benefit from the technology's capabilities and best practices.","Standardized workflows across different companies using the same technology can lead to industry-wide efficiency and easier talent acquisition."],"side_b":["Workflows are built on years of experience and specific business needs. Technology should support, not dictate, these established, effective processes.","Customizing technology to fit existing workflows reduces disruption, improves user acceptance, and avoids costly retraining or loss of productivity."],"question":"Should companies always prioritize adapting their internal workflows to fit new, advanced technologies, or should new technologies be highly customizable to fit existing workflows?"}

💡 Discussion Facilitation Tips

Ask students to provide specific examples from their workplaces when discussing workflow changes or new tools. Encourage the use of cause-and-effect language by asking 'Why did that happen?' or 'What was the result of that decision?' Remind students to actively listen and build on each other's points, using phrases like 'I agree with [name] that...' or 'On the other hand...'

Session Blueprint

Can you share an experience where updating a tool or system in your workplace initially caused more problems than it solved? What was the underlying reason?