People search "twitch chat logs" about 6,600 times a month in the US, and most of them hit the same wall: Twitch has no API that returns chat from the past. Helix doesn't have one, and neither does EventSub. If a message isn't captured while it scrolls by, it's gone for you. So a chat log is something you build going forward: connect, listen, store.
This post builds twitchlog, a small Go program that does exactly that:
-
logconnects to Twitch chat over its IRC WebSocket, joins channels, and writes every message to SQLite, along with subs, raids, timeouts, bans, and room settings -
searchruns full-text queries (FTS5) across everything you've logged -
exportprints a plain-text log for one channel and day, or for one chatter -
statssummarizes a channel: messages per minute, unique chatters, subscriber share, moderation, top Twitch emotes, events -
importrebuilds the database from the raw IRC log, so the raw log is your source of truth -
live(optional) checks which channels are live before you join them
It's five Go files and two dependencies: github.com/coder/websocket for the WebSocket and modernc.org/sqlite, a pure-Go SQLite (no cgo). Everything below ran on November 20, 2026 with Go 1.27.3 on a Linux VM. I logged #xqc and #kaicenat for about 24 minutes in total (12,443 messages). Output blocks are pasted from those runs, except that I replaced chatters' usernames with viewer_NN placeholders.
Read this first. Twitch's terms prohibit scraping. The Twitch Terms of Service (last modified 08/12/2026) say you may not "access any website, server, software application, or other computer resource owned, used, and/or licensed by Twitch, including but not limited to the Twitch Services, by means of any robot, spider, scraper, crawler, or other automated means for any purpose", and the license section excludes "use of any data mining, robots, or similar data gathering or extraction methods". Read those clauses and think about the legal risk for your use case before you run anything here. Prefer the official routes: Twitch documents IRC and EventSub as its chat interfaces, and for new projects it recommends EventSub (
channel.chat.message) with a user access token that has theuser:read:chatscope. The anonymousjustinfanlogin used below is not documented. Chat is also personal data: usernames next to what people said. Keep logs only as long as you need them, don't publish logs of individual people, and check privacy law (GDPR and similar) if you store them. The code redacts messages that moderators delete or that belong to timed-out and banned users, because Twitch removes them from view too.
What you can and can't get
| What | How | Notes |
|---|---|---|
| Live chat, from now on | IRC over WebSocket (wss://irc-ws.chat.twitch.tv:443) or EventSub |
This post uses IRC; EventSub needs a user token |
| Chat from before you connected | No API | Helix and EventSub only deliver new messages |
| Chat replay on VODs | Only through the website's private GraphQL | Not covered here |
| Who is in chat | Helix Get Chatters, needs a moderator token | Anonymous IRC only lists you in NAMES
|
| Timeouts, bans, deleted messages |
CLEARCHAT and CLEARMSG events |
You only see them if you're connected at the time |
| 7TV, BTTV, FFZ emotes | Not in the IRC tags | They arrive as plain words |
Before writing a parser, I tested which transport the VM could reach. The raw IRC ports didn't work from that network: TLS on port 6697 ended with unexpected eof, and a plain connection to 6667 got HTTP/1.1 400 Bad Request from an HTTP proxy. The WebSocket upgrade on port 443 worked first try:
HTTP/1.1 101 Switching Protocols
Connection: upgrade
Upgrade: websocket
That's the main reason to use WebSockets: it's ordinary HTTPS traffic, so it gets through proxies and firewalls that block IRC ports. Twitch shut down the unencrypted ws:// endpoint on port 80 in August 2025, so use wss://.
Setup
mkdir twitchlog && cd twitchlog
go mod init twitchlog
go get github.com/coder/[email protected] modernc.org/[email protected]
The program is split into irc.go (parsing), conn.go (the connection), store.go (SQLite), main.go (commands), and stats.go (the report and the live check). All five are below in full, so you can copy them into one folder and go build.
Step 1: Parse Twitch's IRC lines
Twitch sends IRC with IRCv3 tags in front. A chat message looks like this (shortened, username replaced):
@badge-info=;badges=bronze-chain/1;color=#8A2BE2;display-name=viewer_01;emotes=;first-msg=0;id=00000000-0000-4000-8000-000000000001;mod=0;returning-chatter=0;room-id=71092938;subscriber=0;tmi-sent-ts=1795147987206;user-id=100000001 :viewer_01!viewer_01@viewer_01.tmi.twitch.tv PRIVMSG #xqc :SCHIZO
The tags carry everything useful: a stable message id, the numeric user-id (logins can change, IDs can't), badge-info with subscription months, emotes with character positions, bits, first-msg, reply-parent-msg-id, and the server timestamp tmi-sent-ts in milliseconds. Tag values are escaped (\s is a space, \: is a semicolon), so unescape them. System messages are full of escapes: a raid arrives as system-msg=1\sraiders\sfrom\s...\shave\sjoined!.
irc.go:
package main
import (
"strings"
)
// Message is one parsed IRC line from Twitch, IRCv3 tags included.
type Message struct {
Raw string
Tags map[string]string
Prefix string // "nick!user@host" or "tmi.twitch.tv"
Command string // PRIVMSG, USERNOTICE, CLEARMSG, PING, 001, ...
Params []string
}
// Nick returns the login part of the prefix ("" for server messages).
func (m Message) Nick() string {
if i := strings.IndexByte(m.Prefix, '!'); i > 0 {
return m.Prefix[:i]
}
return ""
}
// Channel returns the first param without '#', if it is a channel.
func (m Message) Channel() string {
if len(m.Params) > 0 && strings.HasPrefix(m.Params[0], "#") {
return m.Params[0][1:]
}
return ""
}
// Trailing returns the last param (the chat text for PRIVMSG).
func (m Message) Trailing() string {
if len(m.Params) == 0 {
return ""
}
return m.Params[len(m.Params)-1]
}
// Parse turns one line into a Message.
// Format: [@tags ][:prefix ]COMMAND [params] [:trailing]
func Parse(line string) Message {
m := Message{Raw: line, Tags: map[string]string{}}
rest := line
if strings.HasPrefix(rest, "@") {
end := strings.IndexByte(rest, ' ')
if end < 0 {
return m
}
for _, kv := range strings.Split(rest[1:end], ";") {
k, v, _ := strings.Cut(kv, "=")
m.Tags[k] = unescapeTag(v)
}
rest = rest[end+1:]
}
if strings.HasPrefix(rest, ":") {
end := strings.IndexByte(rest, ' ')
if end < 0 {
return m
}
m.Prefix = rest[1:end]
rest = rest[end+1:]
}
for rest != "" {
if strings.HasPrefix(rest, ":") {
m.Params = append(m.Params, rest[1:])
break
}
word, after, found := strings.Cut(rest, " ")
if m.Command == "" {
m.Command = word
} else {
m.Params = append(m.Params, word)
}
if !found {
break
}
rest = after
}
return m
}
// IRCv3 tag values escape ';' ' ' '\' CR and LF.
func unescapeTag(v string) string {
if !strings.Contains(v, `\`) {
return v
}
var b strings.Builder
for i := 0; i < len(v); i++ {
if v[i] != '\\' || i == len(v)-1 {
if v[i] != '\\' {
b.WriteByte(v[i])
}
continue
}
i++
switch v[i] {
case ':':
b.WriteByte(';')
case 's':
b.WriteByte(' ')
case 'r':
b.WriteByte('\r')
case 'n':
b.WriteByte('\n')
default: // "\\" and unknown escapes
b.WriteByte(v[i])
}
}
return b.String()
}
Step 2: The connection
The connection has more rules than the parser:
-
Login. Request the
twitch.tv/tagsandtwitch.tv/commandscapabilities, thenPASSandNICK. For read-only access, anyjustinfanplus digits nick works with any password. With an OAuth token you'd sendPASS oauth:<token>and your own login instead. -
One frame, several lines. Twitch's docs warn that a single WebSocket message can hold several IRC lines separated by
\r\n. Split every frame, or you'll parse a batch as one message and drop the rest. -
PING/PONG. Twitch sends
PING :tmi.twitch.tvabout every five minutes and drops you if you don't answer. In my run they arrived at 11:28:37 and 11:33:36. - RECONNECT. Twitch sends this before restarting a server. Reconnect right away.
- Join limits. Twitch's docs say 20 join attempts per 10 seconds for a normal account, and 100 channels joined at the same time. The code waits 11 seconds after every 15 joins.
-
Silent failures. Joining a channel that doesn't exist returns nothing at all: no error, no
NOTICE. I joinedthisuserdoesnotexist_zz9along with the two real channels and only gotJOINconfirmations for the real ones. Check for theJOINecho.
conn.go:
package main
import (
"context"
"errors"
"fmt"
"log"
"math/rand/v2"
"strings"
"time"
"github.com/coder/websocket"
)
var ircURL = "wss://irc-ws.chat.twitch.tv:443" // var so tests can point it elsewhere
// errReconnect means Twitch asked us to move to another server.
var errReconnect = errors.New("server sent RECONNECT")
// session runs one WebSocket connection until it drops or ctx ends.
// Every parsed line goes to out.
func session(ctx context.Context, channels []string, out chan<- Message) error {
dialCtx, cancel := context.WithTimeout(ctx, 15*time.Second)
defer cancel()
c, _, err := websocket.Dial(dialCtx, ircURL, nil)
if err != nil {
return fmt.Errorf("dial: %w", err)
}
defer c.CloseNow()
c.SetReadLimit(1 << 20) // a burst of long messages can exceed the 32 KB default
send := func(line string) error {
wctx, cancel := context.WithTimeout(ctx, 10*time.Second)
defer cancel()
return c.Write(wctx, websocket.MessageText, []byte(line+"\r\n"))
}
// Anonymous, read-only login: any justinfan<digits> nick, any PASS.
nick := fmt.Sprintf("justinfan%d", 10000+rand.IntN(80000))
for _, l := range []string{
"CAP REQ :twitch.tv/tags twitch.tv/commands",
"PASS SCHMOOPIIE",
"NICK " + nick,
} {
if err := send(l); err != nil {
return err
}
}
// JOINs are limited (20 per 10 s for normal users), so pace them.
go func() {
for i, ch := range channels {
if i > 0 && i%15 == 0 {
select {
case <-time.After(11 * time.Second):
case <-ctx.Done():
return
}
}
if err := send("JOIN #" + strings.ToLower(ch)); err != nil {
return
}
}
}()
for {
// Twitch PINGs about every 5 minutes; 6 minutes of silence = dead link.
rctx, cancel := context.WithTimeout(ctx, 6*time.Minute)
_, data, err := c.Read(rctx)
cancel()
if err != nil {
return fmt.Errorf("read: %w", err)
}
// One WebSocket frame can carry several IRC lines.
for _, line := range strings.Split(strings.TrimRight(string(data), "\r\n"), "\r\n") {
if line == "" {
continue
}
m := Parse(line)
switch m.Command {
case "PING":
if err := send("PONG :" + m.Trailing()); err != nil {
return err
}
case "RECONNECT":
return errReconnect
case "NOTICE":
log.Printf("NOTICE %s: %s (msg-id=%s)", m.Channel(), m.Trailing(), m.Tags["msg-id"])
}
select {
case out <- m:
case <-ctx.Done():
return ctx.Err()
}
}
}
}
// run keeps a session alive with exponential backoff until ctx ends.
func run(ctx context.Context, channels []string, out chan<- Message) {
backoff := time.Second
for ctx.Err() == nil {
start := time.Now()
err := session(ctx, channels, out)
if ctx.Err() != nil {
return
}
if time.Since(start) > time.Minute || errors.Is(err, errReconnect) {
backoff = time.Second // it was healthy; reconnect quickly
}
log.Printf("connection ended: %v; reconnecting in %s", err, backoff)
select {
case <-time.After(backoff):
case <-ctx.Done():
return
}
backoff = min(backoff*2, 2*time.Minute)
}
}
ircURL is a var so you can point a test build somewhere unreachable and watch the backoff work. I built one with -ldflags "-X main.ircURL=wss://127.0.0.1:9":
2026/11/20 11:23:37 connection ended: dial: failed to WebSocket dial: failed to send handshake request: Get "https://127.0.0.1:9": dial tcp 127.0.0.1:9: connect: connection refused; reconnecting in 1s
2026/11/20 11:23:38 connection ended: dial: failed to WebSocket dial: failed to send handshake request: Get "https://127.0.0.1:9": dial tcp 127.0.0.1:9: connect: connection refused; reconnecting in 2s
2026/11/20 11:23:40 connection ended: dial: failed to WebSocket dial: failed to send handshake request: Get "https://127.0.0.1:9": dial tcp 127.0.0.1:9: connect: connection refused; reconnecting in 4s
2026/11/20 11:23:44 connection ended: dial: failed to WebSocket dial: failed to send handshake request: Get "https://127.0.0.1:9": dial tcp 127.0.0.1:9: connect: connection refused; reconnecting in 8s
2026/11/20 11:23:52 connection ended: dial: failed to WebSocket dial: failed to send handshake request: Get "https://127.0.0.1:9": dial tcp 127.0.0.1:9: connect: connection refused; reconnecting in 16s
2026/11/20 11:23:57 done: 0 lines, 0 chat messages stored
The backoff resets after a connection that lasted more than a minute, so a normal RECONNECT doesn't leave you waiting two minutes.
Step 3: Store messages, and honor deletions
This is where most chat loggers are sloppy. Three things deserve care.
Moderation. CLEARMSG means a moderator deleted one message (target-msg-id). CLEARCHAT with a target-user-id is a timeout (when ban-duration is present) or a ban, and Twitch hides that user's recent messages. CLEARCHAT with no target clears the whole chat. The code keeps the row (time, user, channel) but empties the text and records why. In 22 minutes of logging I saw 74 CLEARCHAT events and no CLEARMSG at all. Timeouts are how these chats get moderated: 375 of 7,360 #xqc messages (5.1%) ended up redacted.
Emote positions are code points, not bytes. The emotes tag says 425618:2-4 and means characters 2 to 4. With an emoji earlier in the message, slicing the UTF-8 bytes gives you garbage. Slicing a []rune is correct. I checked all 1,702 emote ranges from my first two runs: 3 of them would have come out wrong with byte slicing, all in one message that started with ✌🏾. That's rare, and it's exactly the kind of bug that slips through testing.
Invisible characters. Twitch rejects the same message sent twice in a row, so chat clients append an invisible character to get around it. In my first two runs, 219 of 11,662 messages ended in U+034F (combining grapheme joiner) and 22 in U+E0000. Strip them before storing, or "KEKW KEKW" and "KEKW KEKW\u034f" count as different messages and searches miss.
Twitch's docs also say the server "may also send a message multiple times" if it thinks you didn't get it, so the message id is the primary key and inserts use INSERT OR IGNORE.
store.go:
package main
import (
"database/sql"
"strconv"
"strings"
"time"
_ "modernc.org/sqlite"
)
const schema = `
PRAGMA journal_mode=WAL;
CREATE TABLE IF NOT EXISTS messages (
id TEXT PRIMARY KEY, channel TEXT, room_id TEXT, user_id TEXT, login TEXT,
display_name TEXT, text TEXT, sent_at INTEGER, badges TEXT,
sub_months INTEGER, bits INTEGER, first_msg INTEGER, reply_to TEXT,
emotes TEXT, deleted TEXT);
CREATE INDEX IF NOT EXISTS messages_chan_time ON messages(channel, sent_at);
CREATE INDEX IF NOT EXISTS messages_user ON messages(user_id);
CREATE VIRTUAL TABLE IF NOT EXISTS messages_fts USING fts5(
text, content='messages', content_rowid='rowid');
CREATE TRIGGER IF NOT EXISTS messages_ai AFTER INSERT ON messages BEGIN
INSERT INTO messages_fts(rowid, text) VALUES (new.rowid, new.text);
END;
CREATE TRIGGER IF NOT EXISTS messages_au AFTER UPDATE OF text ON messages BEGIN
INSERT INTO messages_fts(messages_fts, rowid, text) VALUES ('delete', old.rowid, old.text);
INSERT INTO messages_fts(rowid, text) VALUES (new.rowid, new.text);
END;
CREATE TABLE IF NOT EXISTS events (
id TEXT PRIMARY KEY, channel TEXT, kind TEXT, login TEXT, system_msg TEXT, text TEXT,
sent_at INTEGER, months INTEGER);
CREATE TABLE IF NOT EXISTS roomstate (
channel TEXT PRIMARY KEY, room_id TEXT, slow INTEGER, followers_only INTEGER,
emote_only INTEGER, subs_only INTEGER, updated_at INTEGER);`
func openDB(path string) (*sql.DB, error) {
db, err := sql.Open("sqlite", path)
if err != nil {
return nil, err
}
db.SetMaxOpenConns(1)
if _, err := db.Exec(schema); err != nil {
return nil, err
}
return db, nil
}
func atoi(s string) int { n, _ := strconv.Atoi(s); return n }
func sentAt(m Message) int64 {
if ts, err := strconv.ParseInt(m.Tags["tmi-sent-ts"], 10, 64); err == nil {
return ts
}
return time.Now().UnixMilli()
}
// subMonths reads "subscriber/14" out of badge-info.
func subMonths(badgeInfo string) int {
for _, b := range strings.Split(badgeInfo, ",") {
if k, v, ok := strings.Cut(b, "/"); ok && (k == "subscriber" || k == "founder") {
return atoi(v)
}
}
return 0
}
// emoteNames maps the emotes tag ("25:0-4,6-10/1902:12-16") to names.
// Positions count Unicode code points, not bytes, so index into []rune.
func emoteNames(tag, text string) []string {
if tag == "" {
return nil
}
runes := []rune(text)
var names []string
for _, e := range strings.Split(tag, "/") {
_, ranges, _ := strings.Cut(e, ":")
for _, r := range strings.Split(ranges, ",") {
a, b, ok := strings.Cut(r, "-")
s, t := atoi(a), atoi(b)
if !ok || s < 0 || t >= len(runes) || s > t {
continue
}
names = append(names, string(runes[s:t+1]))
}
}
return names
}
// clean drops the invisible characters chat clients append to get past
// Twitch's "identical message" check (U+034F, U+E0000), plus trailing spaces.
func clean(s string) string {
s = strings.NewReplacer("\u034f", "", "\U000E0000", "").Replace(s)
return strings.TrimSpace(s)
}
// apply writes one IRC message into the database inside tx.
// Returns true if it stored a chat message.
func apply(tx *sql.Tx, m Message) (bool, error) {
switch m.Command {
case "PRIVMSG":
text := clean(m.Trailing())
// /me messages arrive wrapped in CTCP ACTION.
if strings.HasPrefix(text, "\x01ACTION ") {
text = "/me " + strings.TrimSuffix(strings.TrimPrefix(text, "\x01ACTION "), "\x01")
}
res, err := tx.Exec(`INSERT OR IGNORE INTO messages VALUES
(?,?,?,?,?,?,?,?,?,?,?,?,?,?,NULL)`,
m.Tags["id"], m.Channel(), m.Tags["room-id"], m.Tags["user-id"], m.Nick(),
m.Tags["display-name"], text, sentAt(m), m.Tags["badges"],
subMonths(m.Tags["badge-info"]), atoi(m.Tags["bits"]), atoi(m.Tags["first-msg"]),
m.Tags["reply-parent-msg-id"], strings.Join(emoteNames(m.Tags["emotes"], m.Trailing()), " "))
if err != nil {
return false, err
}
n, _ := res.RowsAffected() // 0 when Twitch re-sent a message we already have
return n == 1, nil
case "USERNOTICE": // sub, resub, subgift, raid, announcement, ...
months := atoi(m.Tags["msg-param-cumulative-months"])
text := ""
if len(m.Params) > 1 {
text = m.Trailing()
}
_, err := tx.Exec(`INSERT OR IGNORE INTO events VALUES (?,?,?,?,?,?,?,?)`, m.Tags["id"], m.Channel(),
m.Tags["msg-id"], m.Tags["login"], m.Tags["system-msg"], text, sentAt(m), months)
return false, err
case "CLEARMSG": // a moderator deleted one message
_, err := tx.Exec(`UPDATE messages SET text='', emotes='', deleted='deleted'
WHERE id=?`, m.Tags["target-msg-id"])
return false, err
case "CLEARCHAT":
reason := "ban"
if m.Tags["ban-duration"] != "" {
reason = "timeout"
}
if uid := m.Tags["target-user-id"]; uid != "" {
// Twitch hides a timed-out or banned user's recent messages; do the same.
_, err := tx.Exec(`UPDATE messages SET text='', emotes='', deleted=?
WHERE user_id=? AND channel=? AND sent_at<=? AND deleted IS NULL`,
reason, uid, m.Channel(), sentAt(m))
return false, err
}
_, err := tx.Exec(`UPDATE messages SET text='', emotes='', deleted='clear'
WHERE channel=? AND sent_at<=? AND deleted IS NULL`, m.Channel(), sentAt(m))
return false, err
case "ROOMSTATE": // full state on JOIN, partial updates later
_, err := tx.Exec(`INSERT INTO roomstate VALUES (?,?,?,?,?,?,?)
ON CONFLICT(channel) DO UPDATE SET
slow=COALESCE(excluded.slow, slow),
followers_only=COALESCE(excluded.followers_only, followers_only),
emote_only=COALESCE(excluded.emote_only, emote_only),
subs_only=COALESCE(excluded.subs_only, subs_only),
updated_at=excluded.updated_at`,
m.Channel(), m.Tags["room-id"], optInt(m, "slow"), optInt(m, "followers-only"),
optInt(m, "emote-only"), optInt(m, "subs-only"), time.Now().UnixMilli())
return false, err
}
return false, nil
}
// optInt returns nil when the tag is absent, so partial ROOMSTATE updates
// don't overwrite the other settings.
func optInt(m Message, k string) any {
v, ok := m.Tags[k]
if !ok {
return nil
}
return atoi(v)
}
The FTS5 table uses content='messages', so the text isn't stored twice. The update trigger keeps the index in sync when a message is redacted, so a deleted message stops showing up in search.
Step 4: The commands
main.go wires it together. The logger buffers messages and writes them in one transaction per second, or every 1,000 messages. A commit per message adds a disk sync per message, which gets expensive in a busy chat: #kaicenat peaked at 814 messages in one minute during my run.
With -raw, every IRC line also goes to a text file. That file is worth keeping. import replays it into a fresh database, so you can change the schema or fix a bug and rebuild everything. Because of INSERT OR IGNORE, importing the same file twice adds nothing.
main.go:
// twitchlog: log Twitch chat to SQLite over IRC WebSockets, then search,
// summarize and export it. Usage:
//
// twitchlog log -db chat.db -channels xqc,caedrel [-for 30m] [-raw raw.log]
// twitchlog search -db chat.db -q 'NEAR(gta mall, 10)' [-channel xqc] [-user name]
// twitchlog stats -db chat.db -channel xqc [-since 60m]
// twitchlog export -db chat.db -channel xqc|-user name [-day 2026-11-20] [-tz Asia/Saigon]
// twitchlog import -db chat.db -raw raw.log (rebuild from a raw log)
// twitchlog live -channels xqc,caedrel (needs SCRAPEATLAS_API_KEY)
package main
import (
"bufio"
"context"
"database/sql"
"flag"
"fmt"
"log"
"os"
"os/signal"
"strings"
"syscall"
"time"
)
func main() {
if len(os.Args) < 2 {
fmt.Fprintln(os.Stderr, "usage: twitchlog log|search|stats|export|live [flags]")
os.Exit(2)
}
cmd, args := os.Args[1], os.Args[2:]
fs := flag.NewFlagSet(cmd, flag.ExitOnError)
dbPath := fs.String("db", "chat.db", "SQLite file")
channels := fs.String("channels", "", "comma-separated channel logins")
channel := fs.String("channel", "", "one channel login")
dur := fs.Duration("for", 0, "stop after this long (0 = until Ctrl-C)")
raw := fs.String("raw", "", "also append raw IRC lines to this file")
q := fs.String("q", "", "FTS5 query")
user := fs.String("user", "", "filter by login")
since := fs.Duration("since", time.Hour, "stats window")
day := fs.String("day", "", "export one day (YYYY-MM-DD, in -tz)")
tz := fs.String("tz", "UTC", "time zone for export/search output")
fs.Parse(args)
loc, err := time.LoadLocation(*tz)
if err != nil {
log.Fatal(err)
}
if cmd == "live" {
if err := cmdLive(split(*channels)); err != nil {
log.Fatal(err)
}
return
}
db, err := openDB(*dbPath)
if err != nil {
log.Fatal(err)
}
defer db.Close()
switch cmd {
case "log":
err = cmdLog(db, split(*channels), *dur, *raw)
case "import":
err = cmdImport(db, *raw)
case "search":
err = cmdSearch(db, *q, *channel, *user, loc)
case "stats":
err = cmdStats(db, *channel, *since)
case "export":
err = cmdExport(db, *channel, *user, *day, loc)
default:
err = fmt.Errorf("unknown command %q", cmd)
}
if err != nil {
log.Fatal(err)
}
}
func split(s string) []string {
var out []string
for _, c := range strings.Split(s, ",") {
if c = strings.TrimPrefix(strings.ToLower(strings.TrimSpace(c)), "#"); c != "" {
out = append(out, c)
}
}
return out
}
func cmdLog(db *sql.DB, channels []string, dur time.Duration, rawPath string) error {
if len(channels) == 0 {
return fmt.Errorf("-channels is required")
}
ctx, stop := signal.NotifyContext(context.Background(), os.Interrupt, syscall.SIGTERM)
defer stop()
if dur > 0 {
var cancel context.CancelFunc
ctx, cancel = context.WithTimeout(ctx, dur)
defer cancel()
}
var raw *os.File
if rawPath != "" {
f, err := os.OpenFile(rawPath, os.O_CREATE|os.O_APPEND|os.O_WRONLY, 0o644)
if err != nil {
return err
}
defer f.Close()
raw = f
}
msgs := make(chan Message, 4096)
go func() { run(ctx, channels, msgs); close(msgs) }()
// Batch writes: one transaction per second instead of one per message.
tick := time.NewTicker(time.Second)
defer tick.Stop()
var batch []Message
var stored, total int
flush := func() error {
if len(batch) == 0 {
return nil
}
tx, err := db.Begin()
if err != nil {
return err
}
for _, m := range batch {
ok, err := apply(tx, m)
if err != nil {
tx.Rollback()
return fmt.Errorf("%s: %w", m.Command, err)
}
if ok {
stored++
}
}
batch = batch[:0]
return tx.Commit()
}
report := time.Now()
for {
select {
case m, ok := <-msgs:
if !ok {
err := flush()
log.Printf("done: %d lines, %d chat messages stored", total, stored)
return err
}
total++
if raw != nil {
fmt.Fprintln(raw, m.Raw)
}
switch m.Command {
case "JOIN":
log.Printf("joined #%s", m.Channel())
case "PING":
log.Printf("PING from server, answered with PONG")
case "PRIVMSG", "USERNOTICE", "CLEARMSG", "CLEARCHAT", "ROOMSTATE":
batch = append(batch, m)
}
if len(batch) >= 1000 {
if err := flush(); err != nil {
return err
}
}
case <-tick.C:
if err := flush(); err != nil {
return err
}
if time.Since(report) > time.Minute {
log.Printf("%d chat messages stored so far", stored)
report = time.Now()
}
}
}
}
// cmdImport replays a raw IRC log into the database. INSERT OR IGNORE on the
// message id makes it safe to run twice or over overlapping logs.
func cmdImport(db *sql.DB, rawPath string) error {
f, err := os.Open(rawPath)
if err != nil {
return err
}
defer f.Close()
tx, err := db.Begin()
if err != nil {
return err
}
sc := bufio.NewScanner(f)
sc.Buffer(make([]byte, 64*1024), 1<<20)
lines, stored := 0, 0
for sc.Scan() {
lines++
ok, err := apply(tx, Parse(sc.Text()))
if err != nil {
tx.Rollback()
return fmt.Errorf("line %d: %w", lines, err)
}
if ok {
stored++
}
}
if err := sc.Err(); err != nil {
tx.Rollback()
return err
}
log.Printf("imported %d lines, %d new chat messages", lines, stored)
return tx.Commit()
}
func cmdSearch(db *sql.DB, q, channel, user string, loc *time.Location) error {
if q == "" {
return fmt.Errorf("-q is required")
}
rows, err := db.Query(`
SELECT m.channel, m.login, m.sent_at,
snippet(messages_fts, 0, '[', ']', '…', 12)
FROM messages_fts JOIN messages m ON m.rowid = messages_fts.rowid
WHERE messages_fts MATCH ? AND (?='' OR m.channel=?) AND (?='' OR m.login=?)
ORDER BY m.sent_at DESC LIMIT 50`, q, channel, channel, user, user)
if err != nil {
return err
}
defer rows.Close()
n := 0
for rows.Next() {
var ch, login, snip string
var ts int64
if err := rows.Scan(&ch, &login, &ts, &snip); err != nil {
return err
}
fmt.Printf("%s #%s %s: %s\n", time.UnixMilli(ts).In(loc).Format("2006-01-02 15:04:05"), ch, login, snip)
n++
}
fmt.Printf("%d matches (newest first, max 50)\n", n)
return rows.Err()
}
func cmdExport(db *sql.DB, channel, user, day string, loc *time.Location) error {
if channel == "" && user == "" {
return fmt.Errorf("-channel or -user is required")
}
from, to := int64(0), int64(1<<62)
if day != "" {
d, err := time.ParseInLocation("2006-01-02", day, loc)
if err != nil {
return err
}
from, to = d.UnixMilli(), d.AddDate(0, 0, 1).UnixMilli()
}
rows, err := db.Query(`SELECT sent_at, channel, login, text, deleted FROM messages
WHERE (?='' OR channel=?) AND (?='' OR login=?) AND sent_at>=? AND sent_at<?
ORDER BY sent_at`, channel, channel, user, user, from, to)
if err != nil {
return err
}
defer rows.Close()
for rows.Next() {
var ts int64
var ch, login, text string
var deleted sql.NullString
if err := rows.Scan(&ts, &ch, &login, &text, &deleted); err != nil {
return err
}
if deleted.Valid {
text = "<message " + deleted.String + ">"
}
fmt.Printf("[%s] #%s %s: %s\n", time.UnixMilli(ts).In(loc).Format("2006-01-02 15:04:05"), ch, login, text)
}
return rows.Err()
}
Step 5: Stats, and checking who's live
stats.go has the channel report and the optional live command. Emote counts come from the emotes tag, so only Twitch-native emotes are counted. 7TV, BTTV, and FFZ emotes (OMEGALUL, KEKW, ICANT, and friends) are plain words in the text. You could count them by matching against those services' emote lists, but that's a separate project.
Joining channels that aren't live wastes join slots and fills your database with nothing. IRC can't tell you whether a channel is live, and Helix needs an app token for that. I used ScrapeAtlas* for this check. It's a scraping API that covers 50 platforms, and its GET /v1/twitch/profile/basic?handle=... returns isLive and currentViewersCount. If you already have a Helix app token, Get Streams does the same job.
stats.go:
package main
import (
"database/sql"
"encoding/json"
"fmt"
"net/http"
"net/url"
"os"
"sort"
"strings"
"time"
)
func cmdStats(db *sql.DB, channel string, since time.Duration) error {
if channel == "" {
return fmt.Errorf("-channel is required")
}
from := time.Now().Add(-since).UnixMilli()
var n, chatters, subs, firsts, bits int
var t0, t1 sql.NullInt64
err := db.QueryRow(`SELECT COUNT(*), COUNT(DISTINCT user_id),
COALESCE(SUM(sub_months>0),0), COALESCE(SUM(first_msg),0), COALESCE(SUM(bits),0),
MIN(sent_at), MAX(sent_at)
FROM messages WHERE channel=? AND sent_at>=?`, channel, from).
Scan(&n, &chatters, &subs, &firsts, &bits, &t0, &t1)
if err != nil {
return err
}
if n == 0 {
return fmt.Errorf("no messages for #%s in the last %s", channel, since)
}
mins := max(float64(t1.Int64-t0.Int64)/60000, 1)
fmt.Printf("#%s, last %s: %d messages from %d chatters over %.1f min (%.1f msg/min)\n",
channel, since, n, chatters, mins, float64(n)/mins)
fmt.Printf(" from subscribers: %.1f%% first-time chatters: %d bits cheered: %d\n",
100*float64(subs)/float64(n), firsts, bits)
var peakMin int64
var peak int
db.QueryRow(`SELECT sent_at/60000 AS m, COUNT(*) c FROM messages
WHERE channel=? AND sent_at>=? GROUP BY m ORDER BY c DESC LIMIT 1`, channel, from).Scan(&peakMin, &peak)
fmt.Printf(" busiest minute: %s UTC with %d messages\n",
time.UnixMilli(peakMin*60000).UTC().Format("15:04"), peak)
rows, err := db.Query(`SELECT COALESCE(deleted,'kept'), COUNT(*) FROM messages
WHERE channel=? AND sent_at>=? GROUP BY 1 ORDER BY 2 DESC`, channel, from)
if err != nil {
return err
}
var parts []string
for rows.Next() {
var k string
var c int
rows.Scan(&k, &c)
parts = append(parts, fmt.Sprintf("%s=%d", k, c))
}
rows.Close()
fmt.Println(" moderation:", strings.Join(parts, " "))
// Top chatters.
rows, err = db.Query(`SELECT login, COUNT(*) c FROM messages
WHERE channel=? AND sent_at>=? GROUP BY user_id ORDER BY c DESC LIMIT 5`, channel, from)
if err != nil {
return err
}
parts = parts[:0]
for rows.Next() {
var login string
var c int
rows.Scan(&login, &c)
parts = append(parts, fmt.Sprintf("%s (%d)", login, c))
}
rows.Close()
fmt.Println(" top chatters:", strings.Join(parts, ", "))
// Top Twitch emotes (from the emotes tag, so third-party emotes are not counted).
counts := map[string]int{}
rows, err = db.Query(`SELECT emotes FROM messages
WHERE channel=? AND sent_at>=? AND emotes<>''`, channel, from)
if err != nil {
return err
}
for rows.Next() {
var e string
rows.Scan(&e)
for _, name := range strings.Fields(e) {
counts[name]++
}
}
rows.Close()
type kv struct {
k string
v int
}
var top []kv
for k, v := range counts {
top = append(top, kv{k, v})
}
sort.Slice(top, func(i, j int) bool {
if top[i].v != top[j].v {
return top[i].v > top[j].v
}
return top[i].k < top[j].k
})
parts = parts[:0]
for i := 0; i < len(top) && i < 8; i++ {
parts = append(parts, fmt.Sprintf("%s %d", top[i].k, top[i].v))
}
fmt.Println(" top Twitch emotes:", strings.Join(parts, ", "))
// Events (subs, gifts, raids...).
rows, err = db.Query(`SELECT kind, COUNT(*) FROM events
WHERE channel=? AND sent_at>=? GROUP BY kind ORDER BY 2 DESC`, channel, from)
if err != nil {
return err
}
parts = parts[:0]
for rows.Next() {
var k string
var c int
rows.Scan(&k, &c)
parts = append(parts, fmt.Sprintf("%s=%d", k, c))
}
rows.Close()
fmt.Println(" events:", strings.Join(parts, " "))
return nil
}
// cmdLive asks ScrapeAtlas which channels are live right now and prints
// them comma-separated, so you can feed them to `twitchlog log -channels`.
func cmdLive(channels []string) error {
key := os.Getenv("SCRAPEATLAS_API_KEY")
if key == "" {
return fmt.Errorf("set SCRAPEATLAS_API_KEY")
}
var live []string
for _, ch := range channels {
req, _ := http.NewRequest("GET",
"https://api.scrapeatlas.com/v1/twitch/profile/basic?handle="+url.QueryEscape(ch), nil)
req.Header.Set("x-api-key", key)
res, err := http.DefaultClient.Do(req)
if err != nil {
return err
}
var p struct {
DisplayName string `json:"displayName"`
IsLive bool `json:"isLive"`
CurrentViewersCount *int `json:"currentViewersCount"`
Failure *struct {
Code string `json:"code"`
} `json:"failure"`
}
err = json.NewDecoder(res.Body).Decode(&p)
res.Body.Close()
switch {
case err != nil:
return fmt.Errorf("%s: %w", ch, err)
case p.Failure != nil:
fmt.Fprintf(os.Stderr, "%-16s %s\n", ch, p.Failure.Code)
case p.IsLive:
fmt.Fprintf(os.Stderr, "%-16s live, %d viewers\n", ch, *p.CurrentViewersCount)
live = append(live, ch)
default:
fmt.Fprintf(os.Stderr, "%-16s offline\n", ch)
}
}
fmt.Println(strings.Join(live, ","))
return nil
}
Running it
Build it and check who's live:
$ go build -o twitchlog .
$ export SCRAPEATLAS_API_KEY=...
$ ./twitchlog live -channels xqc,caedrel,kaicenat,zackrawrr,shroud,jynxzi,thisuserdoesnotexist_zz9
xqc live, 32822 viewers
caedrel offline
kaicenat live, 30218 viewers
zackrawrr offline
shroud offline
jynxzi offline
thisuserdoesnotexist_zz9 not_found
xqc,kaicenat
Status goes to stderr and the live list goes to stdout, so you can pass it straight in: ./twitchlog log -channels "$(./twitchlog live -channels ...)".
Log for 12 minutes:
$ ./twitchlog log -db chat.db -channels xqc,kaicenat -for 12m -raw raw2.log
2026/11/20 11:24:07 joined #xqc
2026/11/20 11:24:07 joined #kaicenat
2026/11/20 11:25:05 250 chat messages stored so far
2026/11/20 11:26:05 755 chat messages stored so far
2026/11/20 11:27:05 1174 chat messages stored so far
2026/11/20 11:28:05 1711 chat messages stored so far
2026/11/20 11:28:37 PING from server, answered with PONG
2026/11/20 11:29:06 2411 chat messages stored so far
2026/11/20 11:30:07 3106 chat messages stored so far
2026/11/20 11:31:08 3704 chat messages stored so far
2026/11/20 11:32:09 4427 chat messages stored so far
2026/11/20 11:33:09 4994 chat messages stored so far
2026/11/20 11:33:36 PING from server, answered with PONG
2026/11/20 11:34:09 5398 chat messages stored so far
2026/11/20 11:35:09 6106 chat messages stored so far
2026/11/20 11:36:05 done: 7279 lines, 7188 chat messages stored
The first run lasted 10 minutes (4,474 messages) and the second one 12 minutes (7,188). Notice the gap: the first run ended at 11:23:05 and the second one joined at 11:24:07, so about a minute of chat is missing from my logs for good. Nothing can fill it in later. For a real logger, run it as a service and leave it running.
The stats report, with chatters' usernames replaced:
$ ./twitchlog stats -db chat.db -channel xqc -since 3h
#xqc, last 3h0m0s: 7360 messages from 1374 chatters over 26.4 min (279.2 msg/min)
from subscribers: 36.7% first-time chatters: 22 bits cheered: 800
busiest minute: 04:31 UTC with 574 messages
moderation: kept=6982 timeout=375 ban=3
top chatters: viewer_02 (94), viewer_03 (78), viewer_04 (78), viewer_05 (75), viewer_06 (68)
top Twitch emotes: LUL 71, xqcL 40, BatChest 32, DinoDance 9, xqcEZ_TK 9, DinoBusted 8, Kreygasm 8, JinxLUL 7
events: resub=7 viewermilestone=2 sub=2
$ ./twitchlog stats -db chat.db -channel kaicenat -since 3h
#kaicenat, last 3h0m0s: 5083 messages from 1536 chatters over 26.3 min (193.1 msg/min)
from subscribers: 25.8% first-time chatters: 135 bits cheered: 1000
busiest minute: 04:35 UTC with 814 messages
moderation: kept=4952 timeout=128 ban=3
top chatters: viewer_07 (107), viewer_08 (65), viewer_09 (52), viewer_10 (49), viewer_11 (48)
top Twitch emotes: kaicFireMGM 448, <3 62, kaicQM 53, NeedHeal 42, BleedPurpleHD 36, kaicMGM 35, kaicHeart 27, WutFace 24
events: resub=9 sub=8 raid=5 submysterygift=2 subgift=2 viewermilestone=1
The two chats differ in clear ways. #xqc is busier and more of its messages come from subscribers (36.7% vs 25.8%), and moderators hand out a lot of timeouts. #kaicenat has six times as many first-time chatters, more raids, and one channel emote, kaicFireMGM, accounts for about a third of all Twitch emotes used there (448 of 1,370). The "busiest minute" is in UTC (04:35 UTC is 11:35 in Asia/Saigon, where I ran this).
Full-text search uses FTS5 syntax: phrases in quotes, * for prefixes, NEAR(a b, 10) for words close together.
$ ./twitchlog search -db chat.db -q '"gta 6"' -channel xqc -tz Asia/Saigon | tail -6
2026-11-20 11:21:00 #xqc viewer_12: [gta 6] and 2000s youtube in one stream, nice
2026-11-20 11:20:25 #xqc viewer_13: have you seen that mall in [gta 6] yet
2026-11-20 11:17:08 #xqc viewer_14: the [gta 6] has that gta 5 driving wobble screen thing, i…
2026-11-20 11:14:12 #xqc viewer_15: [GTA 6] is kinda mid compared to gta 5
2026-11-20 11:13:19 #xqc viewer_14: why is forsen not streaming [gta 6] yet :(
12 matches (newest first, max 50)
$ ./twitchlog search -db chat.db -q 'NEAR(gta mall, 10)' -tz Asia/Saigon
2026-11-20 11:20:25 #xqc viewer_13: have you seen that [mall] in [gta] 6 yet
1 matches (newest first, max 50)
Pass user input through as-is and FTS5 will choke on punctuation:
$ ./twitchlog search -db chat.db -q 'gta 6?'
2026/11/20 11:42:19 SQL logic error: fts5: syntax error near "?" (1)
If the query comes from users, wrap each word in double quotes before you pass it on.
Exports print one line per message. Redacted messages keep their place in the timeline, so the log stays honest about what happened without repeating what was removed:
$ ./twitchlog export -db chat.db -channel xqc -day 2026-11-20 -tz Asia/Saigon
[2026-11-20 11:20:53] #xqc viewer_16: glass in front
[2026-11-20 11:20:53] #xqc viewer_17: jewelry
[2026-11-20 11:20:53] #xqc viewer_18: All that for 300
[2026-11-20 11:20:53] #xqc viewer_19: <message timeout>
[2026-11-20 11:20:54] #xqc viewer_20: Glass
[2026-11-20 11:20:54] #xqc viewer_21: jewlery
[2026-11-20 11:20:54] #xqc viewer_22: meow
[2026-11-20 11:20:55] #xqc viewer_23: jewelry💎!!
[2026-11-20 11:20:55] #xqc viewer_24: EZ
[2026-11-20 11:20:55] #xqc viewer_25: Omds break the front
[2026-11-20 11:20:55] #xqc viewer_26: smash the glasses
[2026-11-20 11:20:55] #xqc viewer_27: Merch drop #3 is out https://xqc.store/ EZ
Storage is small. The raw logs for all 12,443 messages came to 4.9 MB, and the SQLite file, FTS index included, to 3.4 MB. That's about 400 bytes of raw IRC per message, and a chat at 280 messages a minute would produce roughly 160 MB of raw log per 24 hours of streaming. Compress old raw logs with gzip.
Pitfalls, in one list
- There's no backfill. Every minute your logger is down is lost. Neither IRC nor EventSub has history.
- Raw IRC ports may be blocked. Port 443 WebSockets got through where 6667 and 6697 didn't.
-
Split frames on
\r\n. One WebSocket message can carry several IRC lines. -
Answer
PING, or you get dropped. It comes about every 5 minutes, and a read deadline a bit longer than that catches dead connections. -
A nonexistent channel fails silently. Confirm each
JOINecho. -
Emote ranges are code points. Slice
[]rune, not bytes. - Strip U+034F and U+E0000 before storing or comparing messages.
-
Deduplicate on the message
id. Twitch can send the same message twice, and replays will too. -
Use
user-id, not the login, to group messages by person. Logins can be renamed. - Timeouts redact more than one message. Store the reason, and keep the CLEARCHAT timestamp so a replay doesn't redact messages sent after it.
- Batch your writes. A busy chat broke 800 messages a minute.
- Third-party emotes are invisible to the tags.
- Join limits: 20 joins per 10 seconds, 100 channels at once on a normal account.
Wrapping up
- Twitch chat logs are forward-only. You build them by being connected; no API gives you yesterday's chat.
-
IRC over
wss://on port 443 is the simplest way to read chat, and the anonymous login needs no app registration. EventSub is the documented route Twitch recommends. - Treat moderation as data. Timeouts and deletions arrive as events. Honoring them keeps your log in line with what Twitch shows, and it's the decent thing to do.
-
Keep the raw log. With
import, it's a full rebuild whenever your schema changes.
What would you track in a chat? Emote trends across a season, moderation load per stream, how raids change a chat? If you've built a Twitch chat logger with EventSub instead, I'd like to hear how it went.
*I build it.
Top comments (0)