iOS Packet Tunnel Provider with Local On-Device Server
Asked Answered
G

1

6

I'm using the Network Extension framework provided by Apple to build a packet sniffing/monitoring application similar to Charles Proxy and Surge 4 for iOS.

So far, I have the basic structure of the project up and running with the Main Application triggering the PacketTunnelProvider Extension where I can see packets being forwarded via the packetFlow.readPackets(completionHandler:) method. My background isn't in networking so I'm confused on the basic structure of these kinds of apps. Do they host a server on the device that act as the proxy which intercepts network requests? Could anyone provide a diagram of the general flow of the network requests? I.e. what is the relationship between the Packet Tunnel Provider, Proxy Server, Virtual Interface, and Tunnel?

If these apps do use a local on-device server, how do you configure the NEPacketTunnelNetworkSettings to allow for a connection? I have tried incorporating a local on-device server such as GCDWebServer with no luck in establishing a link between the two.

For example, if the GCDWebServer was reachable at 192.168.1.231:8080, how would I change the code below for the client to communicate with the server?

Main App:

    let proxyServer = NEProxyServer(address: "192.168.1.231", port: 8080)
    
    let proxySettings = NEProxySettings()
    proxySettings.exceptionList = []
    proxySettings.httpEnabled = true
    proxySettings.httpServer = proxyServer
    
    let providerProtocol = NETunnelProviderProtocol()
    providerProtocol.providerBundleIdentifier = self.tunnelBundleId
    providerProtocol.serverAddress = "My Server"
    providerProtocol.providerConfiguration = [:]
    providerProtocol.proxySettings = proxySettings
    
    let newManager = NETunnelProviderManager()
    newManager.localizedDescription = "Custom VPN"
    newManager.protocolConfiguration = providerProtocol
    newManager.isEnabled = true
    saveLoadManager()
    self.vpnManager = newManager

PacketTunnelProviderExtension:

func startTunnel(options: [String : NSObject]?, completionHandler: @escaping (Error?) -> Void) {
  ...
        let settings = NEPacketTunnelNetworkSettings(tunnelRemoteAddress: "127.0.0.143")
        settings.ipv4Settings = NEIPv4Settings(addresses: ["198.17.203.2"], subnetMasks: ["255.255.255.255"])
        settings.ipv4Settings?.includedRoutes = [NEIPv4Route.default()]
        settings.ipv4Settings?.excludedRoutes = []
        settings.dnsSettings = NEDNSSettings(servers: ["8.8.8.8", "8.8.4.4"])

        settings.dnsSettings?.matchDomains = [""]
        self.setTunnelNetworkSettings(settings) { error in
            if let e = error {
                NSLog("Settings error %@", e.localizedDescription)
            } else {
                completionHandler(error)
                self.readPackets()
            }
        }
  ...
}
Gerthagerti answered 22/10, 2020 at 20:33 Comment(0)
C
15

I'm working on the iOS version of Proxyman and my experience can help you:

Do they host a server on the device that acts as the proxy which intercepts network requests?

Yes, you have to start a Listener on the Network Extension (not the main app) to act as a Proxy Server. You can write a simple Proxy Server by using Swift NIO or CocoaAsyncSocket.

To intercept the HTTPS traffic, it's a quite big challenge, but I won't mention here since it's out of the scope.

Could anyone provide a diagram of the general flow of the network requests?

As the Network Extension and the Main app are two different processes, so they couldn't communicate directly like normal apps.

Thus, the flow may look like:

The Internet -> iPhone -> Your Network Extension (VPN) -> Forward to your Local Proxy Server -> Intercept or monitor -> Save to a local database (in Shared Container Group) -> Forward again to the destination server.

From the main app, you can receive the data by reading the local database.

how do you configure the NEPacketTunnelNetworkSettings to allow for a connection?

In the Network extension, let start a Proxy Server at Host:Port, then init the NetworkSetting, like the sample:

    private func initTunnelSettings(proxyHost: String, proxyPort: Int) -> NEPacketTunnelNetworkSettings {
    let settings: NEPacketTunnelNetworkSettings = NEPacketTunnelNetworkSettings(tunnelRemoteAddress: "127.0.0.1")

    /* proxy settings */
    let proxySettings: NEProxySettings = NEProxySettings()
    proxySettings.httpServer = NEProxyServer(
        address: proxyHost,
        port: proxyPort
    )
    proxySettings.httpsServer = NEProxyServer(
        address: proxyHost,
        port: proxyPort
    )
    proxySettings.autoProxyConfigurationEnabled = false
    proxySettings.httpEnabled = true
    proxySettings.httpsEnabled = true
    proxySettings.excludeSimpleHostnames = true
    proxySettings.exceptionList = [
        "192.168.0.0/16",
        "10.0.0.0/8",
        "172.16.0.0/12",
        "127.0.0.1",
        "localhost",
        "*.local"
    ]
    settings.proxySettings = proxySettings

    /* ipv4 settings */
    let ipv4Settings: NEIPv4Settings = NEIPv4Settings(
        addresses: [settings.tunnelRemoteAddress],
        subnetMasks: ["255.255.255.255"]
    )
    ipv4Settings.includedRoutes = [NEIPv4Route.default()]
    ipv4Settings.excludedRoutes = [
        NEIPv4Route(destinationAddress: "192.168.0.0", subnetMask: "255.255.0.0"),
        NEIPv4Route(destinationAddress: "10.0.0.0", subnetMask: "255.0.0.0"),
        NEIPv4Route(destinationAddress: "172.16.0.0", subnetMask: "255.240.0.0")
    ]
    settings.ipv4Settings = ipv4Settings

    /* MTU */
    settings.mtu = 1500

    return settings
}

Then start a VPN,

let networkSettings = initTunnelSettings(proxyHost: ip, proxyPort: port)

// Start
setTunnelNetworkSettings(networkSettings) { // Handle success }

Then forward the package to your local proxy server:

let endpoint = NWHostEndpoint(hostname: proxyIP, port: proxyPort)
self.connection = self.createTCPConnection(to: endpoint, enableTLS: false, tlsParameters: nil, delegate: nil)

    packetFlow.readPackets {[weak self] (packets, protocols) in
        guard let strongSelf = self else { return }
        for packet in packets {
            strongSelf.connection.write(packet, completionHandler: { (error) in
            })
        }

        // Repeat
        strongSelf.readPackets()
    }

From that, your local server can receive the packages then forwarding to the destination server.

Conjunction answered 28/10, 2020 at 9:39 Comment(25)
You're awesome Nghia! I appreciate the detailed response. Would you have any resources that explain how to intercept HTTPS traffic?Gerthagerti
before intercepting HTTPS traffic, I suggest that you should implement a simple HTTP Proxy Server that allows to forward the HTTP/HTTPS traffic, and intercept HTTP (since it's a plain-text).Conjunction
You might checkout mitmproxy doc: docs.mitmproxy.org/stable/concepts-howmitmproxyworksConjunction
I've setup all of your suggestions and mostly everything is working; I'm able to see outgoing packets and parse them. When I run the app, the VPN logo shows but the internet on the device stops working. Do you know why that is? One thing to note, I haven't setup the on device server yet. Would it have something to deal with the DNS settings?Gerthagerti
I'm connecting to the following test server instead where I can see incoming UDP bytes when the app is connected: github.com/davlxd/simple-vpn-demoGerthagerti
@user1068810 Are you trying to forward packets from your extension to your proxy server? I've stopped working on this for now so my implementation isn't finished.Gerthagerti
@user1068810 It's difficult to pinpoint exactly what the issue is on your end but I would take a look at the first link below. From what I recall, you need to call the setTunnelNetworkSettings(_:completionHandler:) followed by the completion handler of the startTunnel method. An example of that can be seen at the second link. #56619965 #56463708Gerthagerti
@user1068810 I only have http working and not https. Once you've correctly configured the packettunnel, you should start receiving packets from the readPackets(completionHandler:) method. From there, you can parse each packet and get the underlying headers and eventually the raw http data.Gerthagerti
@Gerthagerti are you using any parsing library to parse the http data?Inculpable
@Inculpable im using BinarySwift to simplify grabbing bytes of data. If you go this route, you would need to create your own models for IP header, TCP header, etc. github.com/Szaq/BinarySwiftGerthagerti
@Gerthagerti Hi thanks for helping me out in this, if you can share the parsing code, it would be great help, I'm trying but with no success. Only if you can share. Thanks!Inculpable
@Gerthagerti I'm looping through data array and trying to create headermodel object(using BinarySwift) but it seems its returning nothing.Inculpable
@Inculpable could you make a new post about this? I'll help you thereGerthagerti
For HTTP Parser, I highly recommend using this one github.com/nodejs/http-parser . I used to use this library in the production (v1 Proxyman) and it works really wellConjunction
im only using PacketTunnelProviderGerthagerti
@NghiaTran : Which address we should provide here as the proto.serverAddress = "" ,where proto is the object of NETunnelProviderProtocol, as you have specified the local server will start in extension?, i think as mentioned in the docs this address is VPN server address, but that server will be in the extension, Can you please help me that. As i'm not using any external server but local in the app (GCDWebServer).Inculpable
@NghiaTran can you help me with that?Inculpable
Should the local server be started in the init() method or any other delegate?Inculpable
@NghiaTran what will be the iP address used in the protocol configuration in the main as server will be stared in network extension and will that server be pointing to local host?Inculpable
@Gerthagerti after connecting to VPN, my internet is not working too, how did you fix this if you can help me with that?Inculpable
How do we read and write localDb here? What if we just proxy to destination server within network extension?Flea
@Inculpable after connecting to VPN. My internet is not working too, how did you fix this you can help me with that?Humpage
@NghiaTran Apple specifically forbids putting proxy servers in the network extension here: developer.apple.com/documentation/technotes/… "Do not use a packet tunnel provider to host a network listener or proxy server". Am I wrong or this is exactly the approach you are using?Anticipative
@MarcoCarandente thanks for the link. I'm not sure why Apple doesn't recommend this approach since it's the only solution. For example, Charles Proxy iOS app is using this approach.Conjunction
@NghiaTran I believe you can do this in an apple-approved way with a TransparentProxyProvider and userspace network stack. mitmproxy now uses this approach on macOS.Scythia

© 2022 - 2024 — McMap. All rights reserved.