Solved Transfering file via HTTP fails; skips the first 24kb (approx)?

I'm currently working on a plugin that runs a small HTTP server to serve resource packs to the client. However, when I attempt to download it, I notice that the file size is slightly smaller. I (stupidly) tried to compare the two by opening them both up in Notepad++ and noticed that the downloaded version starts around 24kb after the original.

Here's the files via Dropbox (if it helps):
Downloaded version: https://www.dropbox.com/s/7igofm144vvs0c2/downloaded.zip
Original version: https://www.dropbox.com/s/y3hhpnu6hc2yq2p/original.zip

Here's the main HTTP loop code (I am well aware there are resource leaks in it; it's sort of a "do after fix this damn bug"):

Code:java

    @Override
    public void run(){
        socket = null;
        running.set(true);
 
        try{
            httpLog.info("Starting basic HTTP web server on port " + port + "...");
            socket = new ServerSocket(port);
            httpLog.fine("Successfully binded! Awaiting requests.");
        }catch (IOException e){
            e.printStackTrace();
        }
 
        while(running.get()){
            try{
                Socket newConnection = socket.accept();
                BufferedReader reader = new BufferedReader(new InputStreamReader(newConnection.getInputStream()));
                String request = reader.readLine();
 
                if(!request.startsWith("GET")){
                    httpLog.warning("Recieved non-GET HTTP request. Please look into it? " + newConnection.getInetAddress().getHostAddress() + ":" + newConnection.getPort() + " tried " + request);
                    newConnection.close();
                    continue;
                }
 
                request = request.substring(5); // At this point, it should be "somefile.ext HTTP/1.1" or whatever
                request = request.substring(0, request.indexOf(' ')); // Aaaaand now it should be "somefile.ext"
                httpLog.info(newConnection.getInetAddress().getHostAddress() + " is requesting " + request);
                httpLog.info("Outside URL would look like: " + this.getURLDestination(request));
                File target = new File(plugin.getSoundDirectory(), request);
                if(!target.exists()){
                    httpLog.info("Couldn't find the requested file. Terminating connection.");
                    newConnection.close();
                    continue;
                }
 
                FileInputStream stream = new FileInputStream(target);
 
                WritableByteChannel out = Channels.newChannel(newConnection.getOutputStream());
                out.write(ByteBuffer.wrap(getOkayMessage(target).getBytes()));
                stream.getChannel().transferTo(0, target.length(), out);
                out.close();
 
                stream.close();
                reader.close();
            }catch (IOException e){
                e.printStackTrace();
            }
        }
 
        try{
            socket.close();
        }catch (NullPointerException e){
 
        }catch (Throwable e){
            e.printStackTrace();
        }
    }

and getOkayMessage():

Code:java

    public String getOkayMessage(File file){
        return "HTTP/1.1 200 OK\r\nContent-Length: " + file.length() + "\r\nConnection: Keep-Alive\r\nServer: TestHTTP\r\nContent-Type: application/x-zip-compressed\r\n";
    }

 

dont use reader for binary files, read carefully javadocs about that class.

 

I'm not? I'm using it for getting the HTTP method requested.

 

Speaking of reading Javadocs carefully, you should perhaps take an extra look at java.io.File.length(). Also, your choice of IO classes is a bit unorthodox. I can't help but feel like you looked like the character in your signature while writing this

I'm not a huge fan of Java NIO, although NIO.2 is pretty damn sweet, if you're so lucky as to be guaranteed a deployment environment that uses Java 7. If not, and if you don't want to venture into the jungle of Java NIO, just stick with good ol' streams (and take a little time to learn what streams actually are [grab a hold of any functional language and you'll run into them very soon]). Here's a general outline of what you could do:

1. Create a FileInputStream to read the requested file (you've already done this, so yay).
2. Wrap the client output stream in a BufferedOutputStream.
3. Create a byte array to serve as the buffer for reading into and copying from. Give it a sane size, maybe a couple of kilobytes (1024, 2048), but don't go to the order of megabytes - that's not what streams are about.
4. Use the read(byte[], int, int) and write(byte[], int, int) methods of the streams in a while-loop (read the Javadocs carefully, and Google example usages if you're having trouble).
5. Finally, close the streams.

Happy hacking!

 

Didn't work
and yes, I am kinda panicking like the girl in my signature. It's been annoying me for the past 2 days. I've tried your method, the standard while-read-write loop, reading everything into memory and then sending it all at once, nothing works. It keeps skipping the first part of the file; specifically, it skips all the data up until it hits the first line after the new line. The last parts confusing, so here's a picture of what I mean:


Update: It skips the first 284 bytes of a file.
24kb? pfft. I'm terrible at estimations. Tried doing it using a file that weighs only 82 bytes. Upon arrival, it was 0 bytes.

Update 2: It's not 284 bytes; tried a 285 byte file and it still comes up as 0 bytes. Is this a bug within the vanilla client?

 

Are you making sure to flush the BufferedOutputStream after every write? I wrote a tiny little example that works just fine with the zip-file from your Dropbox link. What do you mean by "a bug within the vanilla client"? I don't see any Minecraft-related code at all.

 

Code:java

 try(FileInputStream stream = new FileInputStream(target); BufferedOutputStream output = new BufferedOutputStream(newConnection.socket().getOutputStream());){
int length;
byte[] buffer = new byte[2048];
 
while((length = stream.read(buffer)) != -1){
output.write(buffer, 0, length);
output.flush();
}
}

This is what I'm currently using.
and by bug within the vanilla client, I mean that the client itself is doing this for some reason.

edit: Client gets a "java.util.zip.ZipException: invalid END header (bad central directory offset)"

 

Looks alright, apart from the call to "newConnection.socket()", which I'm sure is a typo. It could be that the client-side isn't actually ready to receive anything yet by the time you start writing (I've only tested locally). Anyway, here's a minimal working example. Fire up your browser on localhost:8080 (or change the port), and it'll send you the file "original.zip", if it is in the same folder as where you invoked the JVM.

Code:java

public class Test {
    private static final int PORT = 8080;
    private static final int BUFFER_SIZE = 2048;
    private static final String HEADER =
        "HTTP/1.1 200 OK\r\n" +
        "Content-Type: application/x-zip-compressed\r\n" +
        "Content-Length: ";
 
    public static void main(String[] args) throws Exception {
        ServerSocket server = new ServerSocket(PORT);
 
        while (true) {
            // Wait for connection
            Socket client = server.accept();
 
            // Store file in a byte array so we can get its size
            FileInputStream fileIn = new FileInputStream(new File("original.zip"));
            ByteArrayOutputStream fileOut = new ByteArrayOutputStream();
            write(fileIn, fileOut);
            fileIn.close();
 
            // Append size of file (in bytes) to header and wrap it up
            String header = HEADER + fileOut.size() + "\r\n\n";
 
            // Grab the client output stream and write the header
            InputStream headerIn = new ByteArrayInputStream(header.getBytes());
            OutputStream clientOut = new BufferedOutputStream(client.getOutputStream());
            write(headerIn, clientOut);
            headerIn.close();
 
            // Finally, write the actual content
            fileOut.writeTo(clientOut);
            clientOut.close();
            fileOut.close();
 
            // And remember to close the client connection
            client.close();
        }
    }
 
    private static void write(InputStream in, OutputStream out) throws IOException {
        byte[] buffer = new byte[BUFFER_SIZE];
        int read;
        while ((read = in.read(buffer)) != -1) {
            out.write(buffer, 0, read);
            out.flush();
        }
    }
}

The downloaded file has the exact same size as the original:

Code:

$ ls -l
total 17280
-rw-r--r--@ 1 [snip]  staff  4415820 May  3 20:34 download.zip
-rw-r--r--@ 1 [snip]  staff  4415820 May  3 20:34 original.zip

 

This works... A bit weird; I'm pretty sure I just screwed up somewhere. Thanks