usockets: How do I specify the external format when I open a socket

Question

I'm trying to connect to a mud client, so I'm using usockets to connect over tcp. But After I write I get a decoding error reading. I have reason to believe the encoding should ascii ,or least use :clrf as the end of line designator, as on the lines I read there is a ^M before the end of line

(let* ((sock (socket-connect "angalon.net" 3011))
       (stream (slot-value sock 'stream)))
  (format stream "guest~%")
  (force-output stream)
  (dotimes (i 40)
    (read-line stream))
  stream)

:UTF-8 stream decoding error on
#<SB-SYS:FD-STREAM
  for "socket 192.168.1.39:65516, peer: 93.174.104.58:3011"
  {1004129903}>:

  the octet sequence #(255 251 1 80) cannot be decoded.
   [Condition of type SB-INT:STREAM-DECODING-ERROR]

I can verify the external format of the stream is indeed :utf-8, but the question is how I specify the external-format of the stream the socket gives me?

(let* ((sock (socket-connect "angalon.net" 3011))
       (stream (slot-value sock 'stream)))
  (stream-external-format stream))
;; => :UTF-8

Shawnshawna

Answer 1

Just by looking at the source for the Clozure CL backend, the external format is hard coded to ccl:*default-external-format* which is UTF-8 in my system. The SBCL backend does not specify an external format, but it probably creates the socket with SBCL defaults, which is again UTF-8. I don't think there's a portable way to change the external format short of modifying usocket.

That said, you could bind sb-impl::*default-external-format* to say :latin-1 before calling socket-connect:

(let* ((sb-impl::*default-external-format* :latin-1)
       (sock (socket-connect "angalon.net" 3011))
       (stream (slot-value sock 'stream)))
   (stream-external-format stream))

;; :LATIN-1

Edit: also take a look at FLEXI-STREAMS. I haven't tested it, but you could convert the stream to a FLEXI-STREAMand specify an external format.

Otoplasty

Thanks! Worked although the ^M persiste. Apparently I cannot assign an external format created by flexi-streams as default-external-format in sbcl which would allow me to specify the end-of-line markers. – Shawnshawna

(MAKE-EXTERNAL-FORMAT) does support specifying EOL type along with encoding. – Otoplasty

I was aware of that, but sb-impl::*default-external-format* doesn't appear to play way with flexi-streams. I've given a detail account of the problem here: #20484776 – Shawnshawna