XMPP FileTransfer - Why does the bytestreams protocol fail?

Question

I'm doing file transfer via XMPP on both Android and iOS to a Spark client (Spark 2.6.3). I have successfully accomplished this on android (due to the nice libraries) but have yet to succeed on iOS. However, on both devices I have noticed that the transfer fails via bytestreams ...

<error code="404" type="CANCEL"><item-not-found xmlns="urn:ietf:params:xml:ns:xmpp-stanzas"/><text xmlns="urn:ietf:params:xml:ns:xmpp-stanzas" xml:lang="en">Could not establish socket with any provided host</text></error>

On Android the transfer completes by resorting to ibb ...

<iq id="DK1IO-19" to="[email protected]/Spark 2.6.3" type="set"><open xmlns="http://jabber.org/protocol/ibb" block-size="4096" sid="jsi_4740809287923906297" stanza="iq"/></iq>

Also, on Android, when I send a file from Spark to my Android app, it seems to work fine via bytesreams.

On iOS therefore I would have to code the ibb connection myself, but why would bytestreams be failing in the first place?

Full Logs (android app) :

05-15 13:42:04.023: I/System.out(523): 01:42:04 PM RCV  (1093423288): <iq id="DK1IO-9" to="[email protected]/123" from="[email protected]/Spark 2.6.3" type="result"><si xmlns="http://jabber.org/protocol/si"><feature xmlns="http://jabber.org/protocol/feature-neg"><x xmlns="jabber:x:data" type="submit"><field var="stream-method"><value>http://jabber.org/protocol/bytestreams</value><value>http://jabber.org/protocol/ibb</value></field></x></feature></si></iq>
05-15 13:42:04.040: I/System.out(523): 01:42:04 PM SENT (1093423288): <iq id="DK1IO-10" to="[email protected]/Spark 2.6.3" type="get"><query xmlns="http://jabber.org/protocol/disco#info"></query></iq>
05-15 13:42:04.060: I/System.out(523): 01:42:04 PM RCV  (1093423288): <iq id="DK1IO-10" to="[email protected]/123" type="result" from="[email protected]/Spark 2.6.3"><query xmlns="http://jabber.org/protocol/disco#info"><identity category="client" name="Smack" type="pc"/><feature var="http://jabber.org/protocol/xhtml-im"/><feature var="http://jabber.org/protocol/muc"/><feature var="http://jabber.org/protocol/bytestreams"/><feature var="http://jabber.org/protocol/commands"/><feature var="http://jabber.org/protocol/si/profile/file-transfer"/><feature var="http://jabber.org/protocol/si"/><feature var="http://jabber.org/protocol/ibb"/></query></iq>
05-15 13:42:04.080: I/System.out(523): 01:42:04 PM SENT (1093423288): <iq id="DK1IO-11" to="beta.example.co.uk" type="get"><query xmlns="http://jabber.org/protocol/disco#items"></query></iq>
05-15 13:42:04.090: I/System.out(523): 01:42:04 PM RCV  (1093423288): <iq type="result" id="DK1IO-11" from="beta.example.co.uk" to="[email protected]/123"><query xmlns="http://jabber.org/protocol/disco#items"><item jid="pubsub.beta.example.co.uk" name="Publish-Subscribe service"/><item jid="broadcast.beta.example.co.uk" name="Broadcast service"/><item jid="search.beta.example.co.uk" name="User Search"/><item jid="conference.beta.example.co.uk" name="Public Chatrooms"/><item jid="proxy.beta.example.co.uk" name="Socks 5 Bytestreams Proxy"/></query></iq>
05-15 13:42:04.100: I/System.out(523): 01:42:04 PM SENT (1093423288): <iq id="DK1IO-12" to="pubsub.beta.example.co.uk" type="get"><query xmlns="http://jabber.org/protocol/disco#info"></query></iq>
05-15 13:42:04.110: I/System.out(523): 01:42:04 PM RCV  (1093423288): <iq id="DK1IO-10" to="[email protected]/123" type="result" from="[email protected]/Spark 2.6.3"><query xmlns="http://jabber.org/protocol/disco#info"><identity category="client" name="Smack" type="pc"/><feature var="http://www.xmpp.org/extensions/xep-0166.html#ns"/><feature var="urn:xmpp:tmp:jingle"/></query></iq>
05-15 13:42:04.130: I/System.out(523): 01:42:04 PM RCV  (1093423288): <iq type="result" id="DK1IO-12" from="pubsub.beta.example.co.uk" to="[email protected]/123"><query xmlns="http://jabber.org/protocol/disco#info"><identity category="pubsub" name="Publish-Subscribe service" type="service"/><feature var="http://jabber.org/protocol/pubsub"/><feature var="http://jabber.org/protocol/pubsub#collections"/><feature var="http://jabber.org/protocol/pubsub#config-node"/><feature var="http://jabber.org/protocol/pubsub#create-and-configure"/><feature var="http://jabber.org/protocol/pubsub#create-nodes"/><feature var="http://jabber.org/protocol/pubsub#delete-nodes"/><feature var="http://jabber.org/protocol/pubsub#get-pending"/><feature var="http://jabber.org/protocol/pubsub#instant-nodes"/><feature var="http://jabber.org/protocol/pubsub#item-ids"/><feature var="http://jabber.org/protocol/pubsub#meta-data"/><feature var="http://jabber.org/protocol/pubsub#modify-affiliations"/><feature var="http://jabber.org/protocol/pubsub#manage-subscriptions"/><feature var="http://jabber.org/protocol/pubsub#multi-subscribe"/><feature var="http://jabber.org/protocol/pubsub#outcast-affiliation"/><feature var="http://jabber.org/protocol/pubsub#persistent-items"/><feature var="http://jabber.org/protocol/pubsub#presence-notifications"/><feature var="http://jabber.org/protocol/pubsub#publish"/><feature var="http://jabber.org/protocol/pubsub#publisher-affiliation"/><feature var="http://jabber.org/protocol/pubsub#purge-nodes"/><feature var="http://jabber.org/protocol/pubsub#retract-items"/><feature var="http://jabber.org/protocol/pubsub#retrieve-affiliations"/><feature var="http://jabber.org/protocol/pubsub#retrieve-default"/><feature var="http://jabber.org/protocol/pubsub#retrieve-items"/><feature var="http://jabber.org/protocol/pubsub#retrieve-subscriptions"/><feature var="http://jabber.org/protocol/pubsub#subscribe"/><feature var="http://jabber.org/protocol/pubsub#subscription-options"/><feature var="http://jabber.org/protocol/pubsub#default_access_model_open"/><feature var="http://jabber.org/protocol/disco#info"/></query></iq>
05-15 13:42:04.160: I/System.out(523): 01:42:04 PM SENT (1093423288): <iq id="DK1IO-13" to="broadcast.beta.example.co.uk" type="get"><query xmlns="http://jabber.org/protocol/disco#info"></query></iq>
05-15 13:42:04.170: I/System.out(523): 01:42:04 PM RCV  (1093423288): <iq type="result" id="DK1IO-13" from="broadcast.beta.example.co.uk" to="[email protected]/123"><query xmlns="http://jabber.org/protocol/disco#info"><identity category="component" type="generic" name="Broadcast service"/><feature var="http://jabber.org/protocol/disco#info"/><feature var="http://jabber.org/protocol/disco#items"/></query></iq>
05-15 13:42:04.190: I/System.out(523): 01:42:04 PM SENT (1093423288): <iq id="DK1IO-14" to="search.beta.example.co.uk" type="get"><query xmlns="http://jabber.org/protocol/disco#info"></query></iq>
05-15 13:42:04.200: I/System.out(523): 01:42:04 PM RCV  (1093423288): <iq type="result" id="DK1IO-14" from="search.beta.example.co.uk" to="[email protected]/123"><query xmlns="http://jabber.org/protocol/disco#info"><identity category="directory" type="user" name="User Search"/><feature var="jabber:iq:search"/><feature var="http://jabber.org/protocol/disco#info"/><feature var="http://jabber.org/protocol/rsm"/></query></iq>
05-15 13:42:04.210: I/System.out(523): 01:42:04 PM SENT (1093423288): <iq id="DK1IO-15" to="conference.beta.example.co.uk" type="get"><query xmlns="http://jabber.org/protocol/disco#info"></query></iq>
05-15 13:42:04.230: I/System.out(523): 01:42:04 PM RCV  (1093423288): <iq type="result" id="DK1IO-15" from="conference.beta.example.co.uk" to="[email protected]/123"><query xmlns="http://jabber.org/protocol/disco#info"><identity category="conference" name="Public Chatrooms" type="text"/><identity category="directory" name="Public Chatroom Search" type="chatroom"/><feature var="http://jabber.org/protocol/muc"/><feature var="http://jabber.org/protocol/disco#info"/><feature var="http://jabber.org/protocol/disco#items"/><feature var="jabber:iq:search"/><feature var="http://jabber.org/protocol/rsm"/></query></iq>
05-15 13:42:04.240: I/System.out(523): 01:42:04 PM SENT (1093423288): <iq id="DK1IO-16" to="proxy.beta.example.co.uk" type="get"><query xmlns="http://jabber.org/protocol/disco#info"></query></iq>
05-15 13:42:04.250: I/System.out(523): 01:42:04 PM RCV  (1093423288): <iq type="result" id="DK1IO-16" from="proxy.beta.example.co.uk" to="[email protected]
05-15 13:42:04.250: I/System.out(523): 01:42:04 PM RCV  (1093423288): xttools.co.uk/123"><query xmlns="http://jabber.org/protocol/disco#info"><identity category="proxy" name="SOCKS5 Bytestreams Service" type="bytestreams"/><feature var="http://jabber.org/protocol/bytestreams"/><feature var="http://jabber.org/protocol/disco#info"/></query></iq>
05-15 13:42:04.330: I/System.out(523): 01:42:04 PM SENT (1093423288): <iq id="DK1IO-17" to="proxy.beta.example.co.uk" type="get"><query xmlns="http://jabber.org/protocol/bytestreams"/></iq>
05-15 13:42:04.341: I/System.out(523): 01:42:04 PM RCV  (1093423288): <iq type="result" id="DK1IO-17" from="proxy.beta.example.co.uk" to="[email protected]/123"><query xmlns="http://jabber.org/protocol/bytestreams"><streamhost jid="proxy.beta.example.co.uk" host="127.0.1.1" port="7777"/></query></iq>
05-15 13:42:04.350: I/System.out(523): 01:42:04 PM SENT (1093423288): <iq id="DK1IO-18" to="[email protected]/Spark 2.6.3" type="set"><query xmlns="http://jabber.org/protocol/bytestreams" sid="jsi_4740809287923906297" mode = "tcp"><streamhost jid="[email protected]/123" host="127.0.0.1" port="7777"/><streamhost jid="proxy.beta.example.co.uk" host="127.0.1.1" port="7777"/></query></iq>
05-15 13:42:04.400: D/dalvikvm(523): GC_CONCURRENT freed 361K, 5% free 9330K/9799K, paused 5ms+4ms
05-15 13:42:09.412: I/System.out(523): 01:42:09 PM RCV  (1093423288): <iq id="DK1IO-18" to="[email protected]/123" from="[email protected]/Spark 2.6.3" type="error"><query xmlns="http://jabber.org/protocol/bytestreams" sid="jsi_4740809287923906297" mode="tcp"><streamhost jid="[email protected]/123" host="127.0.0.1" port="7777"/><streamhost jid="proxy.beta.example.co.uk" host="127.0.1.1" port="7777"/></query><error code="404" type="CANCEL"><item-not-found xmlns="urn:ietf:params:xml:ns:xmpp-stanzas"/><text xmlns="urn:ietf:params:xml:ns:xmpp-stanzas" xml:lang="en">Could not establish socket with any provided host</text></error></iq>
05-15 13:42:09.430: I/System.out(523): 01:42:09 PM SENT (1093423288): <iq id="DK1IO-19" to="[email protected]/Spark 2.6.3" type="set"><open xmlns="http://jabber.org/protocol/ibb" block-size="4096" sid="jsi_4740809287923906297" stanza="iq"/></iq>
05-15 13:42:09.451: I/System.out(523): 01:42:09 PM RCV  (1093423288): <iq id="DK1IO-19" to="[email protected]/123" from="[email protected]/Spark 2.6.3" type="result"/>
05-15 13:42:09.470: I/System.out(523): 01:42:09 PM SENT (1093423288): <iq id="DK1IO-20" to="[email protected]/Spark 2.6.3" type="set"><data xmlns="http://jabber.org/protocol/ibb" seq="0" sid="jsi_4740809287923906297">SGVsbG8sIEFuZHkh</data></iq>
05-15 13:42:09.480: I/System.out(523): 01:42:09 PM RCV  (1093423288): <iq id="DK1IO-20" to="[email protected]/123" from="[email protected]/Spark 2.6.3" type="result"/>
05-15 13:42:09.490: I/System.out(523): 01:42:09 PM SENT (1093423288): <iq id="DK1IO-21" to="[email protected]/Spark 2.6.3" type="set"><close xmlns="http://jabber.org/protocol/ibb" sid="jsi_4740809287923906297"/></iq>
05-15 13:42:09.500: I/System.out(523): 01:42:09 PM RCV  (1093423288): <iq id="DK1IO-21" to="[email protected]/123" from="[email protected]/Spark 2.6.3" type="result"/>

Full Logs (iOS app) :

2012-05-16 10:41:16.988 JabberClient[392:207] RECIEVED IN APP DELEGATE - <iq xmlns="jabber:client" id="5CCE541D-A2D2-4115-8D0F-A683EE9B18AB" to="[email protected]/844b40b3" from="[email protected]/Spark 2.6.3" type="result"><si xmlns="http://jabber.org/protocol/si"><feature xmlns="http://jabber.org/protocol/feature-neg"><x xmlns="jabber:x:data" type="submit"><field var="stream-method"><value>http://jabber.org/protocol/bytestreams</value></field></x></feature></si></iq>
2012-05-16 10:41:16.989 JabberClient[392:207] Attempting XEP65 connection to [email protected]/Spark 2.6.3
2012-05-16 10:41:16.990 JabberClient[392:207] TURNSocket DELEGATE STARTING
2012-05-16 10:41:16.991 JabberClient[392:5307] SENT - <iq type="get" to="beta.example.co.uk" id="24C8A655-1B23-4E49-B0D2-557B863C6EB4"><query xmlns="http://jabber.org/protocol/disco#items"/></iq>
2012-05-16 10:41:16.999 JabberClient[392:5307] RECIEVED IN TURN SOCKET - <iq xmlns="jabber:client" type="result" id="24C8A655-1B23-4E49-B0D2-557B863C6EB4" from="beta.example.co.uk" to="[email protected]/844b40b3"><query xmlns="http://jabber.org/protocol/disco#items"><item jid="pubsub.beta.example.co.uk" name="Publish-Subscribe service"/><item jid="broadcast.beta.example.co.uk" name="Broadcast service"/><item jid="search.beta.example.co.uk" name="User Search"/><item jid="conference.beta.example.co.uk" name="Public Chatrooms"/><item jid="proxy.beta.example.co.uk" name="Socks 5 Bytestreams Proxy"/></query></iq>
2012-05-16 10:41:17.000 JabberClient[392:5307] SENT - <iq type="get" to="proxy.beta.example.co.uk" id="A8D03520-2200-4260-A150-910BC70F794A"><query xmlns="http://jabber.org/protocol/disco#info"/></iq>
2012-05-16 10:41:17.004 JabberClient[392:5007] RECIEVED IN TURN SOCKET - <iq xmlns="jabber:client" type="result" id="A8D03520-2200-4260-A150-910BC70F794A" from="proxy.beta.example.co.uk" to="[email protected]/844b40b3"><query xmlns="http://jabber.org/protocol/disco#info"><identity category="proxy" name="SOCKS5 Bytestreams Service" type="bytestreams"/><feature var="http://jabber.org/protocol/bytestreams"/><feature var="http://jabber.org/protocol/disco#info"/></query></iq>
2012-05-16 10:41:17.005 JabberClient[392:5007] SENT - <iq type="get" to="proxy.beta.example.co.uk" id="50A73193-944D-402D-B2E4-EEACC537B23B"><query xmlns="http://jabber.org/protocol/bytestreams"/></iq>
2012-05-16 10:41:17.008 JabberClient[392:5007] RECIEVED IN TURNSOCKET - <iq xmlns="jabber:client" type="result" id="50A73193-944D-402D-B2E4-EEACC537B23B" from="proxy.beta.example.co.uk" to="[email protected]/844b40b3"><query xmlns="http://jabber.org/protocol/bytestreams"><streamhost jid="proxy.beta.example.co.uk" host="127.0.1.1" port="7777"/></query></iq>
2012-05-16 10:41:17.009 JabberClient[392:5007] SENT - <iq type="set" from="[email protected]/844b40b3" to="[email protected]/Spark 2.6.3" id="5CCE541D-A2D2-4115-8D0F-A683EE9B18AB"><query xmlns="http://jabber.org/protocol/bytestreams" sid="5CCE541D-A2D2-4115-8D0F-A683EE9B18AB" mode="tcp"><streamhost xmlns="http://jabber.org/protocol/bytestreams" jid="proxy.beta.example.co.uk" host="127.0.1.1" port="7777"/></query></iq>
2012-05-16 10:41:27.027 JabberClient[392:540b] RECIEVED IN TURN SOCKET - <iq xmlns="jabber:client" id="5CCE541D-A2D2-4115-8D0F-A683EE9B18AB" to="[email protected]/844b40b3" from="[email protected]/Spark 2.6.3" type="error"><query xmlns="http://jabber.org/protocol/bytestreams" sid="5CCE541D-A2D2-4115-8D0F-A683EE9B18AB" mode="tcp"><streamhost jid="proxy.beta.example.co.uk" host="127.0.1.1" port="7777"/></query><error code="404" type="CANCEL"><item-not-found xmlns="urn:ietf:params:xml:ns:xmpp-stanzas"/><text xmlns="urn:ietf:params:xml:ns:xmpp-stanzas" lang="en">Could not establish socket with any provided host</text></error></iq>
2012-05-16 10:41:27.028 JabberClient[392:540b] TURN Connection failed!

OpenFire XMPP settings -

xmpp.auth.anonymous - true
xmpp.client.compression.policy - disabled
xmpp.client.idle -1
xmpp.client.idle.ping - false
xmpp.domain - beta.example.co.uk
xmpp.enabled - true
xmpp.externalip - proxy.beta.example.co.uk
xmpp.offline.quota - 102400
xmpp.offline.type - bounce
xmpp.proxy.enabled - true
xmpp.proxy.port - 7777
xmpp.proxy.transfer.required - true
xmpp.server.compression.policy - disabled
xmpp.server.socket.active - true
xmpp.session.conflict-limit - 0
xmpp.session.conflict.limit - 0
xmpp.socket.ssl.active - true

Update 19th May: Below are some logs from where Spark successfully sends to android (rather than vice-versa). The streamhost used isn't the proxy! However, I've not yet manage to replicate this on iOS, and really should I? Shouldn't proxy be working?

RCV  (1079219752): <iq id="ktI51-43" to="[email protected]/123" type="set" from="[email protected]/Spark 2.6.3"><query xmlns="http://jabber.org/protocol/bytestreams" sid="jsi_1985730781887292843" mode="tcp"><streamhost jid="[email protected]/Spark 2.6.3" host="10.95.xxx.xx" port="7777"/><streamhost jid="proxy.beta.example.co.uk" host="127.0.1.1" port="7777"/></query></iq>
SENT (1079219752): <iq id="ktI51-43" to="[email protected]/Spark 2.6.3" type="result"><query xmlns="http://jabber.org/protocol/bytestreams"><streamhost-used jid="[email protected]/Spark 2.6.3" /></query></iq>

Note that if I try and send from Spark to my iOS app, when the non-proxy streamhost is used, the connection is refused! (as in point 6 of this SOCKS Protocol Version 5 document). Rep is 5 in the following code, (but it should be 0).

UInt8 ver = [NSNumber extractUInt8FromData:data atOffset:0];
UInt8 rep = [NSNumber extractUInt8FromData:data atOffset:1];

Hosts File : (note: I've replaced some numbers with x's for this Stack Overflow question)

127.0.0.1       localhost
10.95.xxx.xxx   intranet.example.co.uk
255.255.255.255 broadcasthost
::1             localhost
fe80::1%lo0     localhost
10.95.xxx.xxx   beta.example.co.uk

Hartfield

There were some bugs in file transfer in Spark 2.6.3 that may be causing this. I would suggest you try the Beta version of 2.7.x and see if it persists. – Lighterage

Looks like no streamhost could be reached. But only one XMPP stanza is far to less to tell the reason for the failing transfer. – Aneto

Robin : Thanks for your response. Unfortunately I am using a Mac, and Spark 2.7.0 beta doesn't seem to be available for Mac yet. Having said that, the website implies that the big bug fix with file transfer is with IBB. I see no mention of bytestreams. – Hartfield

Flow : Thanks for your response, that's a fair point. I have added the full logs to my question from my android project. – Hartfield

Also, if it does indeed seem like no streamhost could be reached, I don't understand why, and I don't know what step to take next. Could it be relevant that the OpenFire server is on my LAN? – Hartfield

I've added the iOS logs to my question. Also, I've added my OpenFire XMPP Settings in case they are relevant. Is there any other info I should provide? – Hartfield

More updates to my question added. Can anybody help? I think the key issue is the connection is refused, – Hartfield

See if this helps - community.igniterealtime.org/thread/44911 – Zloty

Hi userSeven7s! Thanks for your response! I'm a little confused as to whether your link relates to my problem, mainly because the bug you refer to is for IBB transfers. Shouldn't bytestream transfers still work, or am I misunderstanding something??? Also, if it is still relevant, I am using Spark 2.6.3 and I'm unsure what version of Smack it uses. My android app uses asmack-jse-buddycloud-2010.12.11.jar and again I'm not sure what version of Smack that uses. And iOS is nothing to do with Smack unless I'm mistaken? – Hartfield

Any comments or answers anyone? The Bounty is nearly finished? – Hartfield

The streamhost offered has the local ip address 127.0.0.1 as host which of course cannot work. <streamhost jid="proxy.beta.myCompany.co.uk" host="127.0.1.1" port="7777"/> check if you can configure the host in your Openfire config. – Glottalized

Thankyou for your response Alex. My knowledge of this type of thing (servers, ip addresses, hosts etc) is where I have a lack of knowledge. Can you explain to me what I should be changing the ip address to? – Hartfield

to the public ip address of your streamhost proxy proxy.beta.myCompany.co.uk – Glottalized

Hi Alex. I'm back on this project after a small break. I have now posted my hosts file onto my question. I presume the 10.95.xxx.xxx ip address is the one you believe I should be setting on Openfire? I currently have been unable to spot how or where I can set this? – Hartfield

Aha! In Openfire I can set xmpp.proxy.externalip to either beta.myCompany.co.uk or the ip, and the 404 error no longer appears. However, file transfer doesn't automatically commence still. – Hartfield

I now receive the following error in my OpenFire logs : org.jivesoftware.openfire.filetransfer.proxy.ProxyConnectionManager - Error processing file transfer proxy connection java.io.IOException: Only SOCKS5 supported. (Possibly I need to set up a new stackoverflow question for this.) – Hartfield

Answer 1

I had the wrong setting in my OpenFire properties. xmpp.externalip is wrong. I needed to use xmpp.proxy.enabledip and set this to the public IP.

Hartfield

HI Where can i find this Settings in open fire server@Andy A – Martica

Answer 2

Looks like your server is reporting a steamhost (proxy) of 127.0.0.1, or localhost, which is not correct.

I recently saw this on an openfire server that someone had configured /etc/hosts to point the server name to 127.0.0.1, so that's what it reported to clients when it looked up its own name.

The easiest way to check this will be 'ping <servername>' and 'ping <server FQDN>', and see what IP addresses it comes up with.

For instance, if your server name was beta.example.co.uk, you'd try (While logged into the server):

ping beta ping beta.example.co.uk

if either returns with an IP of 127.0.0.1, check /etc/hosts (or C:\Windows\System32\drivers\etc\hosts if on windows) and make sure beta and beta.example.co.uk are not there (or have the appropriate data if they are there).

Decortication

Hi Zack. Thanks so much for your comment. I've added my hosts file onto the question here. 127.0.0.1 is against localhost. Beta has a different ip address (which I think I needed to add for regular XMPP text messages to work). If I do 'ping beta.myCompany.co.uk' I get regular responses such as ... '64 bytes from 10.95.xxx.xxx: icmp_seq=0 ttl=64 time=3.799 ms'. Does it appear there is still something incorrectly setup? – Hartfield

Answer 3

Yes, file transfer will have this error. Don't use compiled library use the source code of smack for Android called amack/smack it will help you to trace the error. I did use the source code.

https://github.com/rtreffer/smack/blob/master/source/org/jivesoftware/smack/XMPPConnection.java

Hullda