1
00:00:00,300 --> 00:00:03,200
Today's episode is brought to you by
Jellypod AI.

2
00:00:03,640 --> 00:00:10,180
And we've got a handy one from version
2.1.319, which landed November 6,

3
00:00:10,360 --> 00:00:11,100
2026.

4
00:00:11,680 --> 00:00:16,059
It's a new command: slash claude-api
latency-optimize.

5
00:00:17,900 --> 00:00:21,800
If your app calls the Claude API and
people sit there waiting,

6
00:00:22,300 --> 00:00:22,840
this is for you.

7
00:00:23,875 --> 00:00:28,215
The changelog line says it finds where
your Claude API code makes people wait,

8
00:00:28,615 --> 00:00:30,675
and tries faster settings against your
eval.

9
00:00:31,455 --> 00:00:32,715
So it's an interactive skill.

10
00:00:32,955 --> 00:00:36,335
It reads the code that makes your
requests, spots the bottlenecks,

11
00:00:36,495 --> 00:00:39,755
then runs candidate configurations against
your own eval suite.

12
00:00:40,755 --> 00:00:44,315
That last part matters, because most teams
tune latency by gut feel.

13
00:00:45,800 --> 00:00:46,140
Guilty.

14
00:00:46,820 --> 00:00:51,160
I've nudged a setting, watched one request
feel quicker, and called it done.

15
00:00:52,060 --> 00:00:54,280
The thing is, latency isn't one number.

16
00:00:54,820 --> 00:00:59,700
There's time to first token, time to last
token, and output tokens per second.

17
00:01:00,380 --> 00:01:01,980
A voice bot cares about the first.

18
00:01:02,540 --> 00:01:05,860
A job that summarises a financial document
cares about the last.

19
00:01:06,280 --> 00:01:08,180
So you need to know which one hurts.

20
00:01:09,310 --> 00:01:11,150
Right, and each has its own levers.

21
00:01:11,610 --> 00:01:15,330
Prompt caching, so you're not reprocessing
the same system prompt every call.

22
00:01:15,830 --> 00:01:18,010
Effort levels, low, medium, high.

23
00:01:18,410 --> 00:01:20,290
Thinking budgets, or adaptive thinking.

24
00:01:20,910 --> 00:01:22,410
Fast mode on supported models.

25
00:01:22,790 --> 00:01:26,230
And shorter outputs, like structured
outputs instead of chatty prose.

26
00:01:26,790 --> 00:01:30,330
The command's job is checking those
against your code, not guessing.

27
00:01:31,032 --> 00:01:32,712
So how do you actually use it?

28
00:01:33,572 --> 00:01:37,672
You open a terminal in the repo where your
Anthropic SDK code lives,

29
00:01:38,132 --> 00:01:41,312
Python or TypeScript, and where your evals
live too.

30
00:01:41,972 --> 00:01:47,352
Then you just run slash claude-api
latency-optimize.

31
00:01:48,192 --> 00:01:51,992
It looks at where requests pay the first
token and last token penalties.

32
00:01:52,452 --> 00:01:54,612
It checks whether your prompts actually
hit the cache.

33
00:01:55,052 --> 00:01:57,992
It checks whether effort is set too high
for routine tasks,

34
00:01:58,412 --> 00:02:00,472
and whether thinking tokens can be
trimmed.

35
00:02:01,140 --> 00:02:02,420
Then it tries things.

36
00:02:02,660 --> 00:02:04,320
Drop effort from high to low.

37
00:02:04,580 --> 00:02:06,600
Try a smaller model for the easy path.

38
00:02:07,000 --> 00:02:07,980
Turn on fast mode.

39
00:02:08,780 --> 00:02:12,840
And it compares results against your eval,
looking at p50, the median,

40
00:02:13,200 --> 00:02:14,740
and p90, the tail.

41
00:02:16,340 --> 00:02:20,420
That tail is what users actually complain
about, so I'd read p90 first.

42
00:02:21,188 --> 00:02:25,248
Picture a fraud screening call that's set
to high effort by default.

43
00:02:25,868 --> 00:02:30,268
If low effort passes your eval at the same
quality bar, and the p90 drops,

44
00:02:30,628 --> 00:02:31,428
that's a free win.

45
00:02:32,228 --> 00:02:35,708
If it fails, you've learned that cheaply,
before production did.

46
00:02:36,452 --> 00:02:37,432
Now, caveats.

47
00:02:37,852 --> 00:02:38,892
First, cost.

48
00:02:39,592 --> 00:02:43,092
Sweeping across several models and
settings burns API credits.

49
00:02:44,332 --> 00:02:48,512
So keep your eval sample representative
but bounded before you kick off a big run.

50
00:02:49,412 --> 00:02:50,552
Second, quality.

51
00:02:51,092 --> 00:02:54,772
Lowering effort or switching off thinking
can hurt reasoning on edge cases.

52
00:02:55,472 --> 00:02:59,592
If your eval is vague, a faster config can
look fine and still regress.

53
00:03:00,072 --> 00:03:02,632
You want rigid assertions, real oracles.

54
00:03:03,197 --> 00:03:07,817
And third, the tool tunes API levers and
loop efficiency.

55
00:03:08,117 --> 00:03:09,217
It can't fix everything.

56
00:03:09,697 --> 00:03:15,037
Tool lookups, a slow gateway hop, or plain
network jitter can still dominate wall

57
00:03:15,097 --> 00:03:15,697
clock time.

58
00:03:16,137 --> 00:03:17,937
So measure end to end as well.

59
00:03:18,857 --> 00:03:22,717
A few smaller things in 2.1.319 worth a
mention.

60
00:03:23,337 --> 00:03:28,517
Slash tasks now has an Earlier in this
session row listing finished subagents,

61
00:03:28,917 --> 00:03:31,677
so you can reopen one and message it, even
after a restart.

62
00:03:32,177 --> 00:03:36,157
There's a new environment variable,
CLAUDE_CODE_DISABLE_FEATURE_FLAGS,

63
00:03:36,397 --> 00:03:39,257
that stops feature flag fetching without
turning off telemetry,

64
00:03:39,717 --> 00:03:41,677
so every flag uses its built in default.

65
00:03:42,370 --> 00:03:46,730
And the third one's a nice fix: slash
commands with leading whitespace,

66
00:03:46,930 --> 00:03:51,150
tabs, or newlines were going to the model
as plain text instead of running.

67
00:03:51,750 --> 00:03:52,610
That one's bitten me.

68
00:03:53,530 --> 00:03:59,470
So, update, run claude dash dash version
to check, and point the new command at your

69
00:03:59,570 --> 00:04:01,150
slowest endpoint first.

