Automatically read chat text from Minecraft

Question

In Minecraft I was hoping to find a way to read the chat automatically like pictured below

In order to record transactions made in the virtual shop into a PostgreSQL database. Preferably using Python. I do not own the Minecraft server.

My plan is to either find a way to directly read the packets sent from the Minecraft server (preferred for reliability, but of unknown difficulty) or as a backup plan maybe figure out how to screen scrape the text. I've found some resources that would let me change the font to monospaced which would provide a more reliable way to read in the font and I believe create perfectly consistent places on the screen for each character. I could face a direction that is close to black but not quite, but would prefer not to have to. As pictured above you see there are many different colors of font to contend with too.

Even after reducing it as described above, I'm still not sure how to turn it into text using python.

Any tips on my approach? Any hints at how I could read the packets coming from the server? Any tips on scraping the text from my screen?

Crestfallen

You're probably going to want to intercept the traffic that the server receives. Python isn't too feasible a tool for this job (unless you're running a Vanilla server with a Python wrapper) - for more help, check out wiki.vg/Main_Page and #mcdevs on irc.freenode.net. – Clairvoyance

@kuyan: He doesn't own the server, he wants to record things on his client. – Woodwind

maybe this will help ? github.com/mmcgill/mc3p – Aikoail

@JoranBeasley: if mc3p works as described than it should be straightforward to create a log plugin that logs to postgres based on mc3p.plugin.log – Seamaid

I believe MC is written in Java, I totally understand that you asked for a Python solution, but you may want to add a Java tag to this post and get some Java programmers to weigh in on this - maybe there is a minor edit you can do on your end to capture the chat details (and THEN send it to Python). – Cornhusk

here's an example of DBHandler for logging module that sends log messages to database if you want to log directly to a database; though it might be more flexible to log to files first and load to/sync with db later – Seamaid

@kuyan: The page you linked to actually has tools for this job written in Python, so it can't be too infeasible… – Woodwind

scraping the screen probably is unfeasible however ... – Aikoail

Answer 1

First, as kuyan suggested, see http://wiki.vg/Main_Page, which has links to various programs that may be useful, either directly or for source to look at.

For example, under Utilities, the first thing that comes up is a logging proxy.

And a bit down, there's mc3p, the program suggested by Joran Beasley—a Python proxy, with plugin support. It only works up to 1.2.5, but sadimusi/mc3p claims to be a 1.4.2-compatible fork. As J.F. Sebastian says, mc3p has an interface for log plugins, so you can just write one that logs to postgres.

If you want to read the packets yourself, that's not hard. You can write a generic TCP proxy in a few dozen lines of Python—or write one in 2 lines of shellscript around netcat that tees the data to your Python script.

The hard part isn't intercepting the data; it's parsing the protocol. Minecraft probably isn't sending "Nightbane: 1 tnt for 100.000 Dollars each", but something like "offer:Nightbane:1:tnt:100" or "\x13\x09Nightbane\x00\x01\x72\x00\x64". From what the wiki says, the protocol is documented, but poorly, and sometimes inaccurately, and the wiki is sometimes incorrect too, and the official code is very ugly and hard to read. Which means that the best way to figure out the protocol is probably by reading sadimusi/mc3p or one of the other projects like McPacketSniffer or ProtoProxy, at which point you have to ask whether it would be easier to just use that project instead of reimplementing it.

At any rate, scraping the screen should be your last resort.

Woodwind

Answer 2

There’s in fact an even better way to read the chat from Minecraft, and it doesn’t require either screen scraping or packet decoding.

Minecraft automatically writes chat messages (and numerous other things) to log files, both in singleplayer and in multiplayer. On Windows, they are located in %appdata%/.minecraft/logs. Previous logs are compressed using gzip, but the latest session’s log is written to the text file latest.log in realtime. Chat messages contain the text [Client thread/INFO]: [CHAT]. You can either open it as you would with a normal file using:

import os
with open(os.getenv("APPDATA")+"/.minecraft/logs/latest.log", "r") as logfile:
    for line in logfile:
        if "[Client thread/INFO]: [CHAT]" in line:
            print line,

Or if you want to read chat in realtime, you can use the code below, slightly modified from the code from this answer:

import time, os

def follow(thefile):
    thefile.seek(0,2)
    while True:
        line = thefile.readline()
        if not line:
            time.sleep(0.1)
            continue
        yield line

if __name__ == "__main__":
    logfile = open(os.getenv("APPDATA")+"/.minecraft/logs/latest.log", "r")
    loglines = follow(logfile)
    for line in loglines:
        if "[Client thread/INFO]: [CHAT]" in line:
            print line,

Onerous

This works! In 1.15 it's [main/INFO]: [CHAT] instead of [Client thread/INFO]: [CHAT] – Swordsman

Maybe you know how I can split CHAT line to sender, receiver and text? Because every server has different chat format – Dioptric

Answer 3

import time

def follow(thefile):
    thefile.seek(0,2)
    while True:
        line = thefile.readline()
        if not line:
            time.sleep(0.1)
            continue
        yield line

if __name__ == "__main__":
    logfile = open(r"C:\Users\***\AppData\Roaming\.tlauncher\legacy\Minecraft\game\logs\latest.log",)
    loglines = follow(logfile)
    for line in loglines:
        print (line)

This code for 1.19.2 Fabric, it sent message from logs in real time. I use it for position in queue on 2b2t P.s. I am from Russia, sorry for my English))

Chiropody