Perl or Python for a log-shaping pipeline?

01

The task

Read a mixed-format access log from standard input, extract five fields, normalise timestamps to UTC, drop bot traffic by user-agent pattern, and emit newline-delimited JSON — as a single file that a colleague can run without a build step.

Constraints

• Runs on a stock Linux host with no package installation permitted.
• Input is 2–20 GB per run, streamed, never fully in memory.
• The person maintaining it in two years may not be the person writing it.

Scope If your job differs on any of those three constraints, the conclusion below does not transfer. The third constraint in particular is the one that flips the answer most often.

02

Decision summary

For this task the two are close enough that the deciding factor is your environment, not the language. Perl wins when you cannot install anything and the transformation is regex-shaped. Python wins when the output schema is going to grow, or when the maintainer pool is larger. Anyone who tells you one is categorically faster here has not defined the task.

03

Choosing between them

04

Environment and versions

Operating system

Not recorded

CPU

Not recorded

Perl

Not recorded

Python

Not recorded

05

The two implementations

Both satisfy the task statement exactly. Both are written for a competent maintainer rather than for a benchmark. A line-by-line diff between two languages would be noise — the lines were never going to match — so the files are segmented by what each passage is doing instead. Pick a stage and both sides mark the passages that answer it.

#!/usr/bin/perl
use strict;
use warnings;
use JSON::PP ();
use Time::Piece;
my $json = JSON::PP->new->canonical;my $bot  = qr/bot|crawler|spider|slurp/i;my $line_re = qr/^(\S+) \S+ \S+ \[([^\]]+)\] "([^"]*)" (\d{3}) (\S+)/;
sub to_utc {
    my $t = Time::Piece->strptime($_[0], '%d/%b/%Y:%H:%M:%S %z');
    return gmtime($t->epoch)->datetime . 'Z';
}
while (my $line = <STDIN>) {    next if $line =~ $bot;    my ($ip, $ts, $req, $status, $bytes) = $line =~ $line_re or next;
    print $json->encode({
        ip     => $ip,
        ts     => to_utc($ts),
        req    => $req,
        status => 0 + $status,
        bytes  => $bytes eq '-' ? 0 : 0 + $bytes,
    }), "\n";}

#!/usr/bin/env python3
import json, re, sys
from datetime import datetime, timezone
LINE = re.compile(r'^(\S+) \S+ \S+ \[([^\]]+)\] "([^"]*)" (\d{3}) (\S+)')BOT  = re.compile(r'bot|crawler|spider|slurp', re.I)

def to_utc(ts: str) -> str:
    dt = datetime.strptime(ts, "%d/%b/%Y:%H:%M:%S %z")
    return dt.astimezone(timezone.utc).isoformat().replace("+00:00", "Z")

for line in sys.stdin:    if BOT.search(line):
        continue    m = LINE.match(line)
    if not m:
        continue
    ip, ts, req, status, size = m.groups()    print(json.dumps({
        "ip": ip,
        "ts": to_utc(ts),
        "req": req,
        "status": int(status),
        "bytes": 0 if size == "-" else int(size),
    }, sort_keys=True))

Both files are 27 lines. That symmetry is not a trick of formatting — it is what happens when two competent implementations answer the same five questions, and it is the reason a raw line-count comparison would have told you nothing.

06

Results

07

Tradeoff matrix

Five dimensions. Each carries which way it leans, how confident we are, and the basis for that confidence. A dimension with no basis is published as none rather than being quietly dropped.

The lean column carries a glyph as well as a colour, and the confidence column is plain text — nothing in this table depends on you being able to distinguish two hues.

08

Weigh it against your situation

The table above is our evidence. What it is worth depends entirely on which constraints you are actually under, and we cannot see those from here. So set them yourself: mark each dimension irrelevant, relevant or decisive, and the page reports where your weighting lands against our dimensions.

09

Methodology

1. The task statement is the contract. Both implementations must satisfy it exactly; anything either one does extra is out of scope.
2. Code samples are written for readability by a competent maintainer, not for benchmark scores.
3. Dimensions carry an explicit confidence level. none means we have no basis and say so instead of guessing.

10

Limitations

11

Corrections and discussion

If a dimension is wrong, say which one and why. Corrections that name a dimension and cite a source are applied and credited; the reviewed date moves when they are.