Pipeline Office Hours #00002 — Binary Feeds, Multiline Logs & Live Debugging
Expanso
0:00 / 0:00
Pipeline Office Hours #00002 — Binary Feeds, Multiline Logs & Live Debugging
22 просмотра · 11 дней назад
Expanso
146 подписчиков
22 просмотра · 11 дней назад
Our second live Pipeline Office Hours. I work through binary inputs, multiline logs, and throughput tests with Expanso Cloud — and debug the things I break along the way.
What happens:
– receive binary socket input and decompress it inside the pipeline
– use regex boundaries to keep a multiline stack trace together
– inspect decoded output and grouped synthetic log events
– run a load ladder, read the results, and try a JSON transformation
– untangle shell aliases and Python errors, accidentally disconnect my node, and get it running again
– walk through the code so you can run the same tests on your hardware
Recorded live September 10, 2026. The bugs, debugging, and recovery are all part of it. I’ve cut the Twitch screen checks, unrelated screens, and dead space around those switches.
A few details matter: the displayed binary file is reference output; its link to the fresh live run is not established in the recording. The multiline exception is synthetic test data. Benchmark results describe short runs of these workloads on this machine, not a production capacity limit.
Try it:
https://github.com/expanso-io/office-... — session code
https://github.com/expanso-io/benchma... — benchmark harness
https://cloud.expanso.io — Expanso Cloud
https://docs.expanso.io — documentation
https://expanso.io — the product
Chapters:
0:00 Binary feeds and multiline events
1:20 Binary pipeline: socket input and decoding
2:52 Read the binary pipeline
5:39 Inspect decoded reference output
7:03 Multiline logs: one event, several lines
8:55 Start the pipeline and inspect grouped output
12:09 Throughput test: workload and rate ladder
13:01 Results and measurement limits
14:54 Live debugging: shell aliases and Python errors
17:35 I disconnected the node — getting it running again
20:42 JSON transform benchmark results
21:24 Run the benchmark on your hardware
22:22 Bring your next pipeline challenge
Join us:
Slack https://exso.cloud/slack
X https://x.com/expansoio
Bluesky https://bsky.app/profile/expanso.io
LinkedIn / expanso-io
Twitch / expansoio
Bring me your hardest pipeline challenges.