Sign in to follow this  
grumpyOldDude

sending client file to server back to client in Real time

Recommended Posts

Networking sub-forum is generally quiet about now, so better here.

I have been slowly creeping upwards in the networking part of my project.  

The problem I currently have is illustrated in this diagram ... a lot of times my English fails me so I'm guessing the diagram might complement my communications inadequacies. All works fine from stages 1 to 5 in real-time only if I send a saved file from a previous run-time (not the current file just received from client, saved to server HD) back to client. But fails if i try to the send same file received (from the current server run-time) and saved to HD. Is this a limitation of client server - is it the way it is meant to work? or am I doing something wrong?

I would like to send current file just saved in the same server run-time, because the current file would be used to generate more info and data that would be added on (enhancing the puzzle game) ... before it is read,uploaded and sent back to client.  many Thanks

[attachment=35493:client server update.png]

Share this post


Link to post
Share on other sites

Here it is,  many thanks

Included is the client code, just in case you need to check

In the server code, if FILE_RECEIVED (from client) and FILE_TO_SEND are different, it works (server returns the file to client), if they are the same (that is, file_to_send is pointing to file_received) the server doesn't return the file

I've cleaned code of debug prints, so it should be very readable now. The  Android code before the send button code can be ignored, as it is less relevant

server code

public class FileServer {    
      public final static String FILE_RECEIVED ="C:\\root_folder\\QCD_070075c.jpg";
      public final static String FILE_TO_SEND = "C:\\root_folder\\QCD_050050z.jpg"; 
  
    public static void main(String[] args) throws IOException {
        int filesize=2373620;                    //temporarilly hardcoded but large enough
        long start = System.currentTimeMillis();       
        int bytesRead;     
        int current = 0;   
       //===============================================
        FileInputStream fis = null;
        BufferedInputStream bis = null;
        OutputStream os = null;           
        ServerSocket servsock = null;
        Socket sock = null;            
        //==============================================
        InetAddress IP=InetAddress.getLocalHost();    
        servsock = new ServerSocket(57925);      
        System.out.println("IP "+IP.getHostAddress()+"  ***%% :"+servsock.getLocalPort());  
                   
        while (true) {  
            System.out.println("receiving frm client and then send bk...");           
            System.out.println("555_999 Waiting...");           
            sock = servsock.accept();
            System.out.println("Accepted connection : " + sock);
                             
            byte [] mybytearray  = new byte [filesize];
            InputStream is = sock.getInputStream();
            FileOutputStream fos = new FileOutputStream(FILE_RECEIVED); // destination path and name of file
            BufferedOutputStream bos = new BufferedOutputStream(fos);

   //=================== read integer from client ==========
            InputStreamReader isr = new InputStreamReader(is);
            BufferedReader br = new BufferedReader(isr);
            String number = br.readLine();
            System.out.println("integer received from client is "+String.valueOf(number));
   //=======================================================         
            bytesRead = is.read(mybytearray,0,mybytearray.length);
            current = bytesRead;
                                         
            do {
               bytesRead =  is.read(mybytearray, current, (mybytearray.length-current));
               if(bytesRead >= 0) current += bytesRead;
            } while(current < Integer.valueOf(number));

            bos.write(mybytearray, 0 , current);
            bos.flush();
            long end = System.currentTimeMillis();
            System.out.println("AT SERVER: bytesRead  "+bytesRead+"  current  "+current);
            bos.close();

//======================== preparing to Send Data to Client  ==============================              
            // send file
            File myFile = new File (FILE_TO_SEND);
            mybytearray  = new byte [(int)myFile.length()];
            fis = new FileInputStream(myFile);
            bis = new BufferedInputStream(fis);
            bis.read(mybytearray,0,mybytearray.length);
            os = sock.getOutputStream();
  //=========================== send integer to client ===============                   
           OutputStreamWriter osw = new OutputStreamWriter(os);
            BufferedWriter bw = new BufferedWriter(osw);
            number = Integer.toString(mybytearray.length);
            String sendMessage = number + "\n";
            bw.write(sendMessage);
            bw.flush();   
  //========================== send file to client ===================                
            System.out.println("Sending " + FILE_TO_SEND + "(" + mybytearray.length + " bytes)");
            os.write(mybytearray,0,mybytearray.length);
            os.flush();
            System.out.println("number  "+number);
            System.out.println("Done.");
 //           servsock.close();
 //           sock.close(); 
/**/       }
    } 
}

client code

public class FSendfileActivity extends Activity {
    private Socket sock;
    private static final int SELECT_PICTURE = 1;
    private String serverIP = "192.168.1.4";    
    private String selectedImagePath;
    private ImageView img;
    final static String qcd = "qcd";
    String ImageDir2Client;      
    FileOutputStream fos = null;
    BufferedOutputStream bos = null;
//====================
    public  static String   FILE_TO_RECEIVED=null;
    int bytesRead = -1;
    int current = 0; 
     
    public final static int FILE_SIZE = 2373620;
    @Override
    public void onCreate(Bundle savedInstanceState) {
        super.onCreate(savedInstanceState);
        setContentView(R.layout.fsendfile);
 
	    ImageDir2Client = Environment.getExternalStorageDirectory().getAbsolutePath();
	    FILE_TO_RECEIVED = ImageDir2Client + "/clientRootFolder/fromServer000018ccc.jpg";  
	   	     
	    Log.v(qcd, "setup complete ^^^^^ ...");

        img = (ImageView) findViewById(R.id.ivPic);
         
       ((Button) findViewById(R.id.bBrowse)).setOnClickListener(new OnClickListener() {
           public void onClick(View arg0) {
              
                Intent intent = new Intent();
                intent.setType("image/*");
                intent.setAction(Intent.ACTION_GET_CONTENT);
                startActivityForResult( Intent.createChooser( intent, "Select Picture" ), SELECT_PICTURE );
            }
       });
              
       Button send = (Button) findViewById(R.id.bSend);
       final TextView status = (TextView) findViewById(R.id.tvStatus);
       
       send.setOnClickListener(new View.OnClickListener() {
           @Override                     
           public void onClick(View arg0) {
           	new Thread(new Runnable() {                  
                   @Override        
                   public void run() {            
                       try {                                     
                       	    sock = new Socket();     
                            connection(sock,  serverIP, 57925);
                       	
                            File myFile = new File (selectedImagePath); 
                            byte [] mybytearray  = new byte [(int)myFile.length()];
                            FileInputStream fis = new FileInputStream(myFile);
                            BufferedInputStream bis = new BufferedInputStream(fis);
                            bis.read(mybytearray,0,mybytearray.length);
                            OutputStream os = sock.getOutputStream();
            //=========================== send integer to server ===============                   
                            OutputStreamWriter osw = new OutputStreamWriter(os);
                            BufferedWriter bw = new BufferedWriter(osw);
                            String number = Integer.toString(mybytearray.length);
                            String sendMessage = number + "\n";
                            Log.v(qcd, "mybytearray.length    "+mybytearray.length);
                            bw.write(sendMessage);   // send size integer here
                            bw.flush(); 
            //==================================================================                
                          
                        os.write(mybytearray,0,mybytearray.length); // send file
                        os.flush();  
                                          
    //=================  client receiving data ==============================
                 
                        // receive file
                        mybytearray  = new byte [FILE_SIZE];                       
                        InputStream is = sock.getInputStream();                   
                       fos = new FileOutputStream(FILE_TO_RECEIVED);                
                        bos = new BufferedOutputStream(fos);
        //=================== read integer from client ==========                    
                        InputStreamReader isr = new InputStreamReader(is);                   
                       BufferedReader br = new BufferedReader(isr);                
                        number = br.readLine();                 
        //=======================================================         
                        bytesRead = is.read(mybytearray,0,mybytearray.length);     
                       current = bytesRead;
                        do {
                           bytesRead = is.read(mybytearray, current, (mybytearray.length-current));
                           if(bytesRead >= 0) current += bytesRead;
                        } while(current < Integer.valueOf(number));
 
                        bos.write(mybytearray, 0 , current);
                        bos.flush();
                        System.out.println("File " + FILE_TO_RECEIVED
                            + " downloaded (" + current + " bytes read)");
                                                         
   /* */                  sock.close();
                    } catch (UnknownHostException e) {
                        e.printStackTrace();
                    } catch (IOException e) {
                        e.printStackTrace();
                    }  finally{
                        try{
         //                   sock.close();
                        }
                        catch(Exception e){
                            e.printStackTrace();
                        }
                    } 
            	} 
	          }).start();
	        }
	    });
        
    }       
    public static void connection(Socket s,  String serverIP, int port) {
        try {   
        	Log.v(qcd, " before connecting ****...");
        	s.connect(new InetSocketAddress(serverIP, port), 120000);
        	Log.v(qcd, " socket connection DONE!! ");
        } catch (UnknownHostException e) {
            e.printStackTrace();
            Log.v(qcd, " Unknown host..."+e);
        } catch (IOException e) {
            e.printStackTrace();
            Log.v(qcd, " Failed to connect...   "+e);
        }
    }
    public void onActivityResult(int requestCode, int resultCode, Intent data) {
        if (resultCode == RESULT_OK) {
            if (requestCode == SELECT_PICTURE) {
                Uri selectedImageUri = data.getData();
                selectedImagePath = getPath(selectedImageUri);
                TextView path = (TextView) findViewById(R.id.tvPath);
                path.setText("Image Path : " + selectedImagePath);
                img.setImageURI(selectedImageUri);
                Log.v(qcd, selectedImagePath);
            }     
        }
    }

    public String getPath(Uri uri) {
        String[] projection = { MediaStore.Images.Media.DATA };
        Cursor cursor = managedQuery(uri, projection, null, null, null);
        int column_index = cursor.getColumnIndexOrThrow(MediaStore.Images.Media.DATA);
        cursor.moveToFirst();
        return cursor.getString(column_index);
    }
}

Share this post


Link to post
Share on other sites
What is the failure? Error / exception - on which side and what does it say? File received is truncated or incorrect - describe? Transfer too slow for "real time" requirement - how long? Something else?

My first guess is that a file handle remains open from the initial transfer, I don't see any close() calls on the file related objects in the client.

Consider wrapping the file handling code in a "try with resources" block on both the client and server - this makes it easier to get right. You can also do this with the socket if the connection is a once off.

Share this post


Link to post
Share on other sites

What is the failure? Error / exception - on which side and what does it say? File received is truncated or incorrect - describe? Transfer too slow for "real time" requirement - how long? Something else?

It doesn't really crash in that sense, but through debug prints i know it hangs indefinitely in the server code, at the line after receiving the integer file size (before receiving the actual file. I cleaned other debug prints for readability).  It hung indefinitely after this print

InputStreamReader isr = new InputStreamReader(is);
BufferedReader br = new BufferedReader(isr);
String number = br.readLine();
System.out.println("integer received from client is "+String.valueOf(number));

My first guess is that a file handle remains open from the initial transfer, I don't see any close() calls on the file related objects in the client. Consider wrapping the file handling code in a "try with resources" block on both the client and server - this makes it easier to get right. You can also do this with the socket if the connection is a once off.

 

Will try these suggestions now and update (edit) the thread after wards. Some close() calls had previously resulted in connection close() peer to peer exception, so had to remove. But I will close some few objects now and run

Share this post


Link to post
Share on other sites

To take out file io as source of problems, you could save in a buffer, and send that back.

 

Using buffered IO, and then using the socket again assumes the buffered io doesn't eat more input data than exactly that one line, which is kind of dangerous.

Reading a decimal number from bytes isn't very difficult, modulo checking everything to death:

int number = 0;
int length = 0;
while (true) {
    byte k = read_byte(is); // Needs IO exception handling
    if (k == '\n') break;
    // check that k >= '0' and k <= '9'
    number = number * 10 + k - '0';
    // check that number stays in safe ranges
    length = length + 1;
    // check that length stays sane
}
// check that length > 0;

Didn't try to complile it, it may need a cast or an explicit conversion to a digit value.

Share this post


Link to post
Share on other sites

Wow! this client server thing is really getting very crazy, so much inconsistencies in the output. One moment an half way improvement to what I already had is working fine, the next, with little/trivial or no changes at all and the read(byte[] b, int off, int len) blocks continuously, even the previously back steps that had no problems at all is blocked. Almost as deadly and mysterious as the image processing nightmarish coding i encountered several months ago. [ a 'block' is defined in the line in red below and also more fully in the documentation below]

It all has to do with this code (in the server code)

          bytesRead = is.read(mybytearray,0,mybytearray.length);
          current = bytesRead;
         
          do {
             bytesRead =  is.read(mybytearray, current, (mybytearray.length-current));
             if(bytesRead >= 0) current += bytesRead;
          } while(bytesRead > -1);

I barely understand the code and I dont really understand why it blocks here. I read read(byte[] b, int off, int len) doc over and over again. Why do the examples i google online all have the line  current = bytesRead;  Without that line i might have understood the code better. But I wouldn't mind even not understanding it .... as long as it works! And thats thing - it is not consistent. Sometimes the code moves closer to working, in that if i hard-code the correct number of the currently saved bytes in the do-while loop -> ex. while(current < 248843);, it works, but then sometimes even this and the previously working stage blocks indefinitely

read(byte[] b, int off, int len)   documentation

 

int java.io.InputStream.read(byte[] b, int off, int len) throws IOException
  • read
    public int read(byte[] b,
                    int off,
                    int len)
             throws IOException
    Reads up to len bytes of data from the input stream into an array of bytes. An attempt is made to read as many as len bytes, but a smaller number may be read. The number of bytes actually read is returned as an integer.

    This method blocks until input data is available, end of file is detected, or an exception is thrown.

    If len is zero, then no bytes are read and 0 is returned; otherwise, there is an attempt to read at least one byte. If no byte is available because the stream is at end of file, the value -1 is returned; otherwise, at least one byte is read and stored into b.

    The first byte read is stored into element b[off], the next one into b[off+1], and so on. The number of bytes read is, at most, equal to len. Let k be the number of bytes actually read; these bytes will be stored in elements b[off] through b[off+k-1], leaving elements b[off+k] through b[off+len-1] unaffected.

    In every case, elements b[0] through b[off] and elements b[off+len] through b[b.length-1] are unaffected.

    The read(b, off, len) method for class InputStream simply calls the method read() repeatedly. If the first such call results in an IOException, that exception is returned from the call to the read(b, off, len) method. If any subsequent call to read() results in a IOException, the exception is caught and treated as if it were end of file; the bytes read up to that point are stored into b and the number of bytes read before the exception occurred is returned. The default implementation of this method blocks until the requested amount of input data len has been read, end of file is detected, or an exception is thrown. Subclasses are encouraged to provide a more efficient implementation of this method.

    Parameters: b - the buffer into which the data is read. off - the start offset in array b at which the data is written. len - the maximum number of bytes to read. Returns: the total number of bytes read into the buffer, or -1 if there is no more data because the end of the stream has been reached. Throws: IOException - If the first byte cannot be read for any reason other than end of file, or if the input stream has been closed, or if some other I/O error occurs. NullPointerException - If b is null. IndexOutOfBoundsException - If off is negative, len is negative, or len is greater than b.length - off See Also: read()

 

This is bordering insanity, seems like I spent all day chasing my own tail (like I have one)

There isn't an exception or a stacktrace to  read even when I place a try-catch around the code, but it just blocks indefinitely. The doc says

  • This method blocks until input data is available, end of file is detected, or an exception is thrown. 

When I place print out all over the place, i see that - the problem starts with the first read(), it reads way less than half the file size sent from the client. I dont know why. And then the second read() blocks forever. This happens only when I try to read the same file save at the current server run time. As described in the first post - when I read a previously saved file even at the current server in real time, it work fine , but then even this also blocks at times. If you think what i'm writing deosn't make any sense, you're very correct because it really makes no sense. 

At one stage, things was so chaotic I had to revert back to a backup for both server and client code to get going again!

dunno if anyone has some knowledge or experience with this and can help with a FIX/hint/solution.  Many thanks

To take out file io as source of problems, you could save in a buffer, and send that back.

 

Using buffered IO, and then using the socket again assumes the buffered io doesn't eat more input data than exactly that one line, which is kind of dangerous.

Reading a decimal number from bytes isn't very difficult, modulo checking everything to death:

int number = 0;
int length = 0;
while (true) {
    byte k = read_byte(is); // Needs IO exception handling
    if (k == '\n') break;
    // check that k >= '0' and k <= '9'
    number = number * 10 + k - '0';
    // check that number stays in safe ranges
    length = length + 1;
    // check that length stays sane
}
// check that length > 0;

Didn't try to complile it, it may need a cast or an explicit conversion to a digit value.

 Thanks Alberth. Didn't see your post in good time as this problem locked me in my own little world for a long while. So with this way, will I have to write a little function -> read_byte()?

I guess by embedding a Java inbuilt code that reads byte and returns an int for each byte, into read_byte(). So at the completion of the loop within read_byte, the total is returned to 'k',  correct? Since the other data I am sending between client and server code is an image file, would this still work? As I shouldn't be able to test for the numbers between 0-9

Share this post


Link to post
Share on other sites

it reads way less than half the file size sent from the client. I dont know why.

This one is easy. Remember the network code splits your big data block into a lot of small packets and sends each of them to the other side?

 

These packets do not arrive all at the same time, so if you read at the other side, and a bunch arrives, the kernel gives you what it received so far, up to the length you requested. Normally, it is less than what you asked for, unless you start asking for a single byte or so. In any case, your local processing is always faster than the network, so at some point you'll run out of received bytes, and you have to wait again for more.

 

You may think the stream is "<number>" "\n" "<data>" but to the application it is one long stream of bytes without any separation markers whatsoever. Data doesn't arrive in blocks as you wrote it. Big blocks get split, smaller blocks may get merged before sending. It is totally feasible that the receiving end gets "<number>\n<d" and "ata>" or it might get "<numb", "er", ">\n<dat", and "a>", or some other random split on the stream in small packets.

 

As for not getting more after that, until you proof otherwise, my guess is that your buffered io wrappers for reading the number are eating some of your file data. Don't read from different levels of io layers from a single source, it's not reliable. Lower layers may optimize performance by reading more than needed at higher level etc.

And then the second read() blocks forever. This happens only when I try to read the same file save at the current server run time. As described in the first post - when I read a previously saved file even at the current server in real time, it work fine , but then even this also blocks at times. If you think what i'm writing deosn't make any sense, you're very correct because it really makes no sense.

After reading the InputStream.read description, and your buffer loop reader, it starts to make sense, but it's a few steps before you arrive there.

 

The first thing what you must do is to stop using code that you do not fully understand. If you don't entirely understand your "read buffer" loop code, you're coding the higher levels on top of "I think it might do X", for some unknown value of X. Together with the inherently random nature of network IO wrt timing, speed, and bandwidth, you get what you experience, seemingly random behavior that works and then breaks again.

The only way forward is to exactly know what X is. You do that by writing your own "read buffer" loop from the InputStream.read documentation. That way, you know the purpose of each and every statement, you know exactly what to expect from that low level function, and you can stack the higher level code on top of known functionality.

So as homework, I'd suggest you write your own "read data into buffer" loop, using only the description of InputStream.read. Afterwards, compare notes, where does your implementation work differently from the $random website one? My guess is you'll do the "-1" case differently, in particular, after receiving all data that you need.

 

The second thing you must do is to stop thinking in server-only and client-only. It seems to me you're coding a server, and you;re coding a client, then run them both and hope it works. In networking, failure and blocking can have 3 causes. A) Server code can be wrong, B) Client code can be wrong, C) Either side can get blocked or die or whatever, due to things that the other side does or does not do. The 3rd item is what is causing your blocking.

To get a handle on this, you can draw the interactions between the server and the client with a message sequence chart

https://en.wikipedia.org/wiki/Message_Sequence_Chart

 

That page has a lot more stuff than you need though. Basically, you draw two vertical lines next to each other, one is labeled "server" and the other is labeleled "client". Time increases vertically down. With arrows from one line to the other, you can show what messages (what data) you send from one end to the other. I'd start after accepting the connection, and up to and including closing the connection (since it's a relevant event that you send by closing the socket, and that you receive with the -1 return value from read).

You'll end up with some horizontal arrows from the client to the server (server receiving length and the file), and then some arrows from client to server (client receiving length and the file), and then 'close connection' arrows at the end. Technically, the arrows are not horizontal (network transfer takes time), but you're not interested in precise timing, just order of messages being exchanged.

If all is well, at this point, you understand that the "-1" case in the google code waits for closing the connection. Relate that to the diagram, and you'll find how things block indefinitely.

 

So with this way, will I have to write a little function -> read_byte()?

It's the elementary InputStream.read function, except for the closed-connection case. I had hoped you'd understand that, but I guessed wrong.

Note this is also $random website code, which above I said you shouldn't use without understanding. Analyze it, and understand how and why it works. Then rebuild it. Know X, instead of guessing what X might be.

Share this post


Link to post
Share on other sites

Keep in mind nothing in networking is going to be "real time" really. You're basically sending and receiving packages through the mail and are responsible for making sure you send and receive all the pieces and also handle all the code related to waiting for them.

As an example if your server is sending a file to a client then the client is going to have to communicate with the server so that both parties know they are in the state of about to transfer a file, then you'd have to client sit there and wait(a specified maximum amount of time usually) for the pieces of the file to be received.

Usually you'll want to code with the mindset of worst case scenario. You might get a really tiny file in 10k pieces, it might take multiple seconds to transfer, the other end could disconnect partway through sending. Network code is very error prone without a lot of safeguards simply because there are so many things that can go wrong.

Share this post


Link to post
Share on other sites

This one is easy. Remember the network code splits your big data block into a lot of small packets and sends each of them to the other side?   These packets do not arrive all at the same time, so if you read at the other side, and a bunch arrives, the kernel gives you what it received so far, up to the length you requested. Normally, it is less than what you asked for,

 

After reading the InputStream.read description, and your buffer loop reader, it starts to make sense, but it's a few steps before you arrive there.   The first thing what you must do is to stop using code that you do not fully understand. If you don't entirely understand your "read buffer" loop code, you're coding the higher levels on top of "I think it might do X", for some unknown value of X. Together with the inherently random nature of network IO wrt timing, speed, and bandwidth, you get what you experience, seemingly random behavior that works and then breaks again. The only way forward is to exactly know what X is. You do that by writing your own "read buffer" loop from the InputStream.read documentation. That way, you know the purpose of each and every statement, you know exactly what to expect from that low level function, and you can stack the higher level code on top of known functionality. So as homework, I'd suggest you write your own "read data into buffer" loop, using only the description of InputStream.read. Afterwards, compare notes, where does your implementation work differently from the $random website one? My guess is you'll do the "-1" case differently, in particular, after receiving all data that you need.   The second thing you must do is to stop thinking in server-only and client-only. It seems to me you're coding a server, and you;re coding a client, then run them both and hope it works. In networking, failure and blocking can have 3 causes. A) Server code can be wrong, B) Client code can be wrong, C) Either side can get blocked or die or whatever, due to things that the other side does or does not do. The 3rd item is what is causing your blocking. To get a handle on this, you can draw the interactions between the server and the client with a message sequence chart

 

once again your assistance has helped solved this problem.

The key was (1) file-data come in smaller chunks (packets) and (2) fully Understanding the code and thus the need for two read().and current = bytesRead; line code. And if instead of 'current' it was named 'chunk', i would have understood it better

The knowledge of 1. made understanding the read()s code very simple. The first read() was to accept the first chunk that was then used to calculate the remaining bytes for the 3rd parameter of read(). And inside the do-while loop, bytesRead continues to add up to small chunks, which goes back to calculate the remainder length-size to be read until finished.

Once I knew this it wasn't to difficult to find where the problem was. the BufferedInputStream object -> bis wasn't closed. If the jpeg file was already saved, bis doesn't need to be closed. But if the server is writing the file at run time, it can't complete the write without bis closed. So once I closed bis, I was able to send the same file save at the server at run time. Even if it read the file in one big swoop bis would still need to be closed, but understanding the concept deeper was the key that made realise closing bis needed was fix.

ripoff gave a hint on this but joining the dots of these hints bit by bit to get a proper overview is not always possible for me

What still annoys me though is every time I try to make another step forward i always encounter a snag. Right now, I can send files from client to server, save, and send the same back to client. So in the server code, after sending a jpg file, i now also sent a text file with String data to client. But when I opened the text file at the client (on my android phone), it didn't contain the string-text. It opens well (which means it wasn't corrupted or something like that) but it just didn't have any string-text data in it that was in the original text file from server.

Its really annoying but this however is where i am stuck at again, why is the text-file empty when I open it at the client device? 

My additional text file with string text.

This is the additional code, the rest is as in the code above (in post 3). Many thanks anyone who helps

 Server code
            ...
            ...
         File pathPlusfile = new File("C:\\rootFolde\\renderData.txt");
         appendToFile( pathPlusfile, "This my crap string text here just for test", 31  );
                
//==================   Send Data to Client  ============================================              

          mybytearray  = new byte [(int)pathPlusfile.length()];
          fis = new FileInputStream(pathPlusfile);
          bis = new BufferedInputStream(fis);
          bis.read(mybytearray,0,mybytearray.length);
          os = sock.getOutputStream();
  
//========================== send file to client ===================                
          os.write(mybytearray,0,mybytearray.length);
          os.flush();
          bis.close();
          ...
          ...

   static BufferedWriter  bufferedWriter = null;

    public static void appendToFile( File myPathPlusFile, String S, int num  ){
 	   
  	   try{
  		  bufferedWriter = new BufferedWriter(new FileWriter(myPathPlusFile, true));
  		  bufferedWriter.write( S ); 
  	          bufferedWriter.newLine();
  		  bufferedWriter.append( S ); 
  		  bufferedWriter.append( "     " ); 
  		  bufferedWriter.append( Integer.toString(num) ); 
  	      bufferedWriter.newLine();
  	      bufferedWriter.flush();
  	   }
  	   catch (IOException e){
  	      e.printStackTrace();
  	   }
    	  closeFile(  );
     }

public static void closeFile(  ){
    try
       bufferedWriter.close();
    }
    catch (IOException e)
    {
       e.printStackTrace();
    }

}

 

I don't mind posting questions where i am stuck at but when i get stuck at every single point I try to make progress it makes me look so dumb :wub:


Moving this to Networking. Please don't post in other forums just because one is quiet; it's not meant to be a real-time chat system but a categorised forum.

 

[Note: the stuff below is not written with a bad attitude or hash tone. Its just me giving my good and valid reason]

There is a very good reason for this

1. I agree this is a voluntary thing and not everyone can be here all the time, so my mentioning of this forum being quiet over the weekend wasn't a complain in any sense. Rather I appreciate the great assistance and help experienced people give those of us with less skills and experience 

2. Because of my day job (weekdays) I have more time to work on my project during weekends than during weekdays.  (I still have time during the week but i have more opportunity of doing marathon hours on my project at weekends). There are also lots of interruptions for me during weekdays

3. I had placed threads here before on Friday night and only one person replied to it, but I needed more than what they gave but I never had one till mid-monday, whereas there are lots of other users who would have helped but don't just visit this sub-forum that much.

And that makes a waste of my optimsed periods of weekends

Alberth who had been extremely helpful even mentioned it in the other thread that he doesn't visit here that much. 

Thanks for your little note, networking forum is not visited regularly by me, so I missed the next problem

 

So I wasn't expecting a chatroom kind of conversation rather I was giving more people who can help the opportunity to see my thread at the time when I am also able to work on my project and their advice the most

4. More so the thread can still be classified as programming in general, so it doesn't break anything by placing this under general programming. Its not as wild as placing a game design question, Lounge question, visual arts question or business and law question in the general programming forum

5. Other mods often see my client server question there a few times and have been ok with it.  

6. The advantage of forums is the different global time zones and this means when one is sleeping others can be awake It. It is also a fact that the ratio of visits during the weekends of GP forum to here is probably roughly 10:1 (where 1 here means one or two people) thus it will be counter productive for a weekend project-person to stick here because of the mild deviation between general programming and network programming

7. And I forgot to mention that you didn't even put the suitcase moved-post icon there, This thread just "disappears".  So maybe other people I'm trying to reply to wouldn't even see this post

So in conclusion my need to place networking code threads in GP forum is, - I want to  maximise (and not waste) my weekends. not that i was expecting a chat service 

(I'm mostly a weekend project person [though i also have a lot of interrupted times during the week] and I gained a lot this past weekend and don't want to miss that in future because of this kind of bureaucracy. That's why I've gone at length to make this case. But it is all with respect, not personal) 

Share this post


Link to post
Share on other sites

Is this a limitation of client server - is it the way it is meant to work? or am I doing something wrong?


The ONLY thing the client knows is the bytes sent by the server. Nothing else. It doesn't know if your server is the same machine, or a virtual machine in a server room downtown, or a smart monkey generating packets by quickly transcribing hex to/from slips of paper.

Other than that, the recommendation I can give is to print and verify EVERY return value from EVERY function. Both on the server and client.
For example, if you get an error sending "the same" file back, is that because of permissions? Does your server still hold the file open for writing, and trying to re-open it for reading fails with a permission error, for example?
Do you get the number of bytes you expect in each call to receive? If not, what is your fallback strategy?

In general, network protocols will want to store incoming data into some kind of buffer, and then look to see if there's a full packet in that buffer yet. If so, read it out, and remove it from the buffer (save anything that's not part of the package you processed!) If not, read some more, and try again. This is the #1 important pattern to use when talking to a network!

Share this post


Link to post
Share on other sites

So in conclusion my need to place networking code threads in GP forum is, - I want to  maximise (and not waste) my weekends. not that i was expecting a chat service 
(I'm mostly a weekend project person [though i also have a lot of interrupted times during the week] and I gained a lot this past weekend and don't want to miss that in future because of this kind of bureaucracy. That's why I've gone at length to make this case. But it is all with respect, not personal)


You're missing one crucial thing; the forum is for everyone. So while you might be right in thinking "I will get more help if I post in General Programming", other people may be thinking "I'm not interested in networking questions; why are they here in General when they have their own subforum"? Other people may also, in future, do a search of this specific subforum to find useful threads on a similar topic, but this thread wouldn't have shown up as it was in the wrong place. And some people with specialist knowledge only view certain subforums, so they will never see your post if it is in General. Hplus0603, for example, hasn't posted outside this subforum for years.

The different subforums are there for a reason. I appreciate it's frustrating to not get help in a timely manner when we can't spend much time on our projects, but it's in everyone's general interest to keep things organised.

 

As for your actual (most recent) question: the code is too cluttered now to give any serious feedback on it. We can't go comparing multiple snippets to try and work out how to piece them together. You say you're sending a file which isn't getting written on the client, but the code you've posted shows it getting written on the server, or at least isn't clear about where that file-writing code lives. I suggest tidying up the formatting, reducing or removing all the static variables (for example, 'appendToFile' should have its own BufferedWriter, not overwrite some shared static), remove any code that isn't essential for diagnosing the current problem, add the logging/checking that Hplus0603 recommended, and then post it up if necessary.

Share this post


Link to post
Share on other sites
But when I opened the text file at the client (on my android phone), it didn't contain the string-text. It opens well (which means it wasn't corrupted or something like that) but it just didn't have any string-text data in it that was in the original text file from server.

Baesides dumping info on number of bytes etc received, also split file-io problems from network-io problems by adding a (temporary, perhaps) buffer.

Get stuff from the network to the buffer, and dump the buffer to the screen, or verify it has the text/data you expect, or whatever it takes to verify the data is unchanged[1] in that buffer.

Any problems with files not working are then already shown not to be caused by network communication.

 

Edit: [1] means "unchanged wrt what you expect to receive"

Edited by Alberth

Share this post


Link to post
Share on other sites

]I appreciate it's frustrating to not get help in a timely manner when we can't spend much time on our projects


In addition: :If the only way you can think of for debugging a program is "ask on a forum," then you need to learn more about various debugging techniques.
Debugging techniques may include:
- turning on all compiler warnings and paying attention to the output
- printing inputs and outputs
- stepping through code in a debugger
- using a static code analyzer and paying attention to its output
- commenting out various bits of code and verify assumptions about behavior
- copy and pasting small snippets of code to small external programs to verify behavior
- (re-)reading the documentation about the various functions used and thinking about every word in the described behavior

Share this post


Link to post
Share on other sites

reducing or removing all the static variables (for example, 'appendToFile' should have its own BufferedWriter, not overwrite some shared static),

I can see what you are saying, but I was forced to make 'appendToFile' static because 'main(String[] args)' is static. If I don't a compile error is flagged

Cannot make a static reference to the non-static method appendToFile(File, String, int) from the type 
FileServer 

And obviously removing static from main() doesn't run

Error: Main method is not static in class server.FileServer, please define the main method as:
   public static void main(String[] args) 

That's a good point, could be the reason for the file at client not containing the string ...  and I need to investigate. 

If Java is one of the languages you code with, then do you know how I can make appendToFile non-static when the compiler is forcing me to make it static?

 

the code is too cluttered now to give any serious feedback on it. We can't go comparing multiple snippets to try and work out how to piece them together.

 

I wasn't sure which was better between posting all code -where new code is mixed with old or isolate new code and post it on its own. I opted for the latter but now seems the former was better. Ideally reposting all code and highlighting new code by changing its text colour would be best. I raised this issue on CSI forum some time ago - so hopefully Khawk is doing something about that. In the mean time I will repost the code and edit this post to update

 

]I appreciate it's frustrating to not get help in a timely manner when we can't spend much time on our projects

In addition: :If the only way you can think of for debugging a program is "ask on a forum," then you need to learn more about various debugging techniques.
Debugging techniques may include:
- turning on all compiler warnings and paying attention to the output
- printing inputs and outputs
- stepping through code in a debugger
- using a static code analyzer and paying attention to its output
- commenting out various bits of code and verify assumptions about behavior
- copy and pasting small snippets of code to small external programs to verify behavior
- (re-)reading the documentation about the various functions used and thinking about every word in the described behavior

 

+1

Share this post


Link to post
Share on other sites

"I was forced to make 'appendToFile' static because 'main(String[] args)' is static" - This gets into basic Java programming things rather than networking, but there are several concepts tied up in this 'static' business:

  1. you have a static main() function as the entry point to your program, which is forced on you by Java;
  2. You're expected to then create objects and call methods on them, and they don't have to be static (and usually shouldn't be), and this means they can reference non-static variables that are members of that object

But more importantly, regarding "static BufferedWriter bufferedWriter" - it doesn't need to be static, and it doesn't need to be declared outside the functions at all. Your appendToFile completely overwrites whatever was stored in that variable, and then completely flushes the object at the end, so there's no need for it to exist after the function completes. So, it should be a local variable.

You might want to spend some time reading some Java fundamentals just to make sure you have all this clear in your head before proceeding. It's not directly related to your problem but you'll fix the problem more quickly when you fully understand all the code around it.

Share this post


Link to post
Share on other sites

Finally fixed! But it would be a muted happy bunny this time because of the long painful time it took. But I'm still more than relieved though

It had to do a few unclosed objects in the server code and also not being closed at the right places. This might seem like "why didn't think of this before?".  Well it was closing stream objects that caused the "SocketException: Connection reset by peer: socket write error" that caused me as much grieve previously. So its a delicate balance, understanding what's really going on is very crucial, so you need to close at the RIGHT place in the code on both the server and client side. Misplacing or not closing/closing too early would causing blocking, connection reset or other errors. At times it may seem logical to close an object that receives after 'writing' the received file, but then it blocks. And you remove the close() and it breaks - so what to do? Well Close after 'writing' the send bytes not after writing the received bytes  

It turns out sending and receiving Jpeg files didn't pick up all the flaws in my previous code. So when I assumed "you can now send and receive .jpg files - so every thing is sorted... I should also be able to send text file with data from server to client with no problems, after all a file is a file, right?" Well except that the data didn't appear at the client received text file 

So who knows, maybe when I try to move up another gear yet more flaws would be revealed. 

Anyway Many thanks to Kylotan, hplus0603, Albert and satharis. All your suggestions have helped

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

Sign in to follow this