Why is my TCP output SO LAGGY?

  • Replies:1
Ben Hutchinson
  • Forum posts: 13

May 2, 2016, 11:47:57 PM via Website

Ok, so I have a FLIR One infrared camera on my android phone, and in the callback function for that device (the one that gets the pixel data) I have this code, that creates a new temporary thread via the AsyncTask.execute command, to send that pixel data over an already established TCP connection.

The code for AsyncTask.execute I'm using looks like this:

    final byte StreamImageBytes[] = renderedImage.pixelData();
    AsyncTask.execute(new Runnable() {
        @Override
        public void run() {
            SendImageOverTCP(StreamImageBytes);
        }
    });

The SendImageOverTCP function looks like this:

private void SendImageOverTCP(final byte Pixels[]){
    try{
        DataOutputStream TCPOutputStream = new DataOutputStream(socket.getOutputStream());
        TCPOutputStream.writeInt(SwapBytesInt(StreamMode));
        TCPOutputStream.write(Pixels);
    }catch(Exception e){}
}

By the way "socket" is the name of the TCP socket object, which was initialized in another piece of code, which I've not included here. StreamMode is a variable that is set elsewhere, and is a value simply used by the receiver to tell it what type of image is contained in each frame that is being streamed to it.

I'm stuck as what to do next, because at first I thought my lagging might be from not properly flushing the data, so I tried:
socket.getOutputStream().flush();

and I also tried:
TCPOutputStream.flush();

Neither of these worked to improve the performance of my TCP connection. By the way, the image data being sent is a 160x120 image, with 2bytes per pixel (16-bit grayscale values). So this is 160*120*2 = 38400 bytes per image. And the framerate is 8.8 frames per second, and there are 8 bits per byte. So the bit rate is then 38400*8.8*8 = 2,703,360 bits per second, which is slightly less than 3Mbps. Considering that my wireless LAN in my house has 50Mbps speed, the video stream data rate of 3Mbps should be EASY for it. There's no reason for it to lag.

Even stranger, the lagging isn't consistent. Sometimes it works fine, but then at other times it slows way down (sometimes even pauses for up to 10 seconds), and then later it rushes to catch up to real-time again. I thought it might have to do with the number of threads that my phone's CPU supports (I've got a Motorola Droid Maxx, and have no idea how many CPU cores it has, or how many simultaneous threads it supports). This app I'm working on uses 3 threads during TCP streaming. One is the UI thread, one is the FLIR One infrared camera's own thread, and one is the temporary thread created with AsyncTask.execute that is used to send data over the TCP connection (I did this, because I heard it's good to have network operations on their own thread). But if the CPU only supports up to 2 threads, then it's going to have to juggle around the threads if there's 3 or more, and that might be what's causing the problem. Thinking that might be the problem, I decided to call SendImageOverTCP directly from the camera's callback function (that runs in the camera's thread ). This didn't solve anything either. That simply causes the camera preview image in the app to lag whenever the TCP sending lags, and to speed up whenever the TCP sending speeds up. So avoiding creating a separate thread for TCP sending didn't solve the problem.

If anybody here knows what might be the problem, please let me know. Thanks in advance.

— modified on May 2, 2016, 11:52:12 PM

Reply
Ben Hutchinson
  • Forum posts: 13

May 3, 2016, 9:12:31 AM via Website

I figured out what's wrong. I think there's a problem with the throughput of the router itself (possibly because of the distance between my computer and the router). After installing Virtual Router software on my PC, and allowing my Android phone to connect directly to my PC, I'm getting a perfect connection, with absolutely no lag, even using the larger image size 320x240 (which at the same frame-rate and bit-depth, has a bit-rate of just under 11Mbps).

Reply