Sign in to follow this  
grumpyOldDude

How to unblock a server wait (in code) without stopping the server run

Recommended Posts

A few things that I'm not clear about. The first is serious, the second is a mystery and the third is an ideal state that i would like my code to run at

1.  Read() function

public int read(byte[] b, int off, int len) throws IOException

  • This method blocks until input data is available, end of file is detected, or an exception is thrown.  


Read() can block for any reason, it might be because faults in server code, user send no data at all or sends corrupted data.

If server code is well written and clear any bugs, the other two could still cause a block. And if you have many players chances are that someone somewhere will inadvertently do something stupid every so often that would cause a server block. I called this one serious (at the first line) because anytime there is a block, I've had to restart server. But it is meant to run continuously. And if the read block assumes data is still coming so it waits and blocks forever, disrupting and blocking other players.

The do-while/read() code is below, what check (code) can I add to the code that would timeout the block, without causing an exception and without needing to restart the server.

          bytesRead = is.read(mybytearray,0,mybytearray.length);
           current = bytesRead;
   
          do {
             bytesRead =  is.read(mybytearray, current, (mybytearray.length-current));
             if(bytesRead >= 0) current += bytesRead;
        } while(current < Integer.valueOf(number));         
 

2. second issue is a bit of a mystery. 

In a thread, the code is supposed to run sequentially. But a line not reach can cause a issue at an earlier line. Again this happened specifically with blocking. Not closing an object in a latter code line (not reached yet) can cause a block in an earlier read() line (more often in server code) 

 

3. My last post in the previous thread I mentioned all was now working fine. Well that was until I decided to send and receive jpeg file integers and text file with data continuously between client and server without closing either. (One cycle here means a client->server->client data transfer). It runs guaranteed in the first cycle. By the third cycle (sometimes second cycle) the application is forced stopped at the end of the run.

I don't know what causes it. The forced stopped produced no crash dump. If it did I don't know where the exception is dumped (definitely not on the console or logcat), so I can't find clues as to what the cause is. But I do need to fix this

In trying to blindly fix this (in the client code), i'm guessing some objects had been closed in the wrong order or not closed at all. But all my attempts to tweak things so far has either resulted in an exception or a block. In fact this 3rd issue is the main reason I was aware of the second issue above 

How do I solve this - as in call sequences, close order, object life-time freed?

I suppose the first and third issues are the core issues i need to solve. The second is like a 3b to 3

Any ideas how I can solve the first and third problems

Many Thanks for any help    

server code

public class FileServer {                 
    public final static  String   FILE_RECEIVED ="C:\\rootFolder\\NEW_070075n.jpg";
   
  public static  void main(String[] args) throws IOException {
   
      int bytesRead;     
      int current = 0;   
     //===============================================
      FileInputStream fis = null;
      BufferedInputStream bis = null;
      OutputStream os = null;           
      ServerSocket servsock = null;
      Socket sock = null;            
      //==============================================
      InetAddress IP=InetAddress.getLocalHost();  
      servsock = new ServerSocket(57925);      
      System.out.println("IP "+IP.getHostAddress()+"  ***%% :"+servsock.getLocalPort());  
                 
      while (true) {  
          System.out.println("555_999 Waiting...");           
          sock = servsock.accept();
          System.out.println("Accepted connection : " + sock);
                           
          
          InputStream is = sock.getInputStream();
          FileOutputStream fos = new FileOutputStream(FILE_RECEIVED); 
          BufferedOutputStream bos = new BufferedOutputStream(fos);
 //=================== read integer from client ==========
          InputStreamReader isr = new InputStreamReader(is);
          BufferedReader br = new BufferedReader(isr);
          String number = br.readLine();
          System.out.println("integer received from client is "+String.valueOf(number));
           byte [] mybytearray  = new byte [Integer.valueOf(number)];
 //=======================================================         
          bytesRead = is.read(mybytearray,0,mybytearray.length);
           current = bytesRead;
          System.out.println("1  bytesRead  "+bytesRead+"  mybytearray.length  "+mybytearray.length);
         
          do {
             bytesRead =  is.read(mybytearray, current, (mybytearray.length-current));
             if(bytesRead >= 0) current += bytesRead;
          } while(current < Integer.valueOf(number));         

	   
          bos.write(mybytearray, 0 , current);
          bos.flush();
          bos.close();
// ======================== write to-be-rendered data to text file ====================== 
                          
          File pathPlusfile = new File("C:/rootFolder/renderData4.txt");
	  appendToFile( pathPlusfile, "Hello, home joel888", 20999  );
                              
//======================== preparing to Send Data to Client  ==============================              
/**/  //==================   Send Data in text file to Client  ============================================              
          // send file   
//          File myFile = new File (FILE_TO_SEND);     
//          mybytearray  = new byte [(int)myFile.length()];
//          fis = new FileInputStream(myFile);
//========================= send jpeg image file to client ===============================		  
          mybytearray  = new byte [(int)pathPlusfile.length()];
          fis = new FileInputStream(pathPlusfile);
          bis = new BufferedInputStream(fis);
          bis.read(mybytearray,0,mybytearray.length);
          os = sock.getOutputStream();
//=========================== send integer to client ===============                   
          OutputStreamWriter osw = new OutputStreamWriter(os);
          BufferedWriter bw = new BufferedWriter(osw);
          number = Integer.toString(mybytearray.length);
          String sendMessage = number + "\n";
          bw.write(sendMessage);
          bw.flush();   
//========================== send file to client ===================                
          os.write(mybytearray,0,mybytearray.length);
          os.flush();
          System.out.println("number  "+number);
          System.out.println("Done.");
          bis.close();
          fis.close();  
          os.close();
          fos.close();        
          is.close(); 

          closeFile(  );               
//           servsock.close();
//           sock.close(); 
     }
  }
  BufferedReader bufferedReader = null;
  String stringObjectData = "";
  public int  numFromFile = 0;

     static BufferedWriter  bufferedWriter = null;
    public static  void appendToFile( File myPathPlusFile, String S, int num  ){
 	   
  	   try{
  		  bufferedWriter = new BufferedWriter(new FileWriter(myPathPlusFile, true));
//  		  bufferedWriter.write( S );   
//  	      bufferedWriter.newLine();
  		  bufferedWriter.append( S ); 
  		  bufferedWriter.append( "     " ); 
  		  bufferedWriter.append( Integer.toString(num) ); 
  	      bufferedWriter.newLine();
  	      bufferedWriter.flush();
  	   }
  	   catch (IOException e){
  	      e.printStackTrace();
  	   }
    	//  closeFile(  );
     }
    
	 public static  void closeFile(  ){
 	   try{
 //	      bufferedWriter.flush();
 	      bufferedWriter.close();
 	   }
 	   catch (IOException e)
 	   {
 	      // TODO Auto-generated catch block
 	      e.printStackTrace();
 	   }
	}
}


Client code

public class FSendfileActivity extends Activity {
    private static final int SELECT_PICTURE = 1;
    private Socket sock;
    private String serverIP = "192.168.1.4";    
    private String selectedImagePath;
    private ImageView img;
    final static String qcd = "qcd";
    String ImageDir2Client;      
    FileOutputStream fos = null;
    BufferedOutputStream bos = null;
//====================
    public  static String   FILE_TO_RECEIVED=null;
    int bytesRead = -1;
    int current = 0;                 
    
    public final static int FILE_SIZE = 2373620;
    @Override
    public void onCreate(Bundle savedInstanceState) {
        super.onCreate(savedInstanceState);
        setContentView(R.layout.fsendfile);    

	    ImageDir2Client = Environment.getExternalStorageDirectory().getAbsolutePath();                    
	    FILE_TO_RECEIVED = ImageDir2Client + "/Server root/fromServer000019eee.txt";   
	   

        img  = (ImageView) findViewById(R.id.ivPic);  
         
       ((Button) findViewById(R.id.bBrowse)).setOnClickListener(new OnClickListener() {
           public void onClick(View arg0) {
              
                Intent intent = new Intent();
                intent.setType("image/*");
                intent.setAction(Intent.ACTION_GET_CONTENT);
                startActivityForResult( Intent.createChooser( intent, "Select Picture" ), SELECT_PICTURE );
            }
       });
              
       Button send = (Button) findViewById(R.id.bSend);
       final TextView status = (TextView) findViewById(R.id.tvStatus);
       
       send.setOnClickListener(new View.OnClickListener() {
           @Override                     
           public void onClick(View arg0) {
           	new Thread(new Runnable() {                  
                   @Override        
                   public void run() {            
                       try {                                     
                       	sock = new Socket();     
                       	connection(sock,  serverIP, 57925);
                       	
                            File myFile = new File (selectedImagePath); 
                            byte [] mybytearray  = new byte [(int)myFile.length()];
                            FileInputStream fis = new FileInputStream(myFile);
                            BufferedInputStream bis = new BufferedInputStream(fis);
                            bis.read(mybytearray,0,mybytearray.length);
                            OutputStream os = sock.getOutputStream();
            //=========================== send integer to server ===============                   
                            OutputStreamWriter osw = new OutputStreamWriter(os);
                            BufferedWriter bw = new BufferedWriter(osw);
                            String number = Integer.toString(mybytearray.length);
                            String sendMessage = number + "\n";
                          
                            bw.write(sendMessage);   // send size integer here
                            bw.flush(); 
            //==================================================================                
                            os.write(mybytearray,0,mybytearray.length); // send file
                            os.flush();  
                                          
    //=================  client receiving data ==============================
 /* */            
                        // receive file
                        mybytearray  = new byte [FILE_SIZE];
                        InputStream is = sock.getInputStream();
                       fos = new FileOutputStream(FILE_TO_RECEIVED);
                        bos = new BufferedOutputStream(fos);
        //=================== read integer from client ==========
        
                        InputStreamReader isr = new InputStreamReader(is);
                       BufferedReader br = new BufferedReader(isr);
                        number = br.readLine();
        //=======================================================         
                        bytesRead = is.read(mybytearray,0,mybytearray.length);
                       current = bytesRead;

                        do {
                           bytesRead = is.read(mybytearray, current, (mybytearray.length-current));
                           if(bytesRead >= 0) current += bytesRead;
                        } while(current < Integer.valueOf(number));
                                 
                        bos.write(mybytearray, 0 , current);
                        bos.flush();
                                                             
                        br.close();  
                        bis.close();       
              //***          bw.close();
                        fos.close();          //STILL EXPERIMENTING WHICH ORDER/MISSING CLOSE()
              //***          bos.close();     //WOULD FIX THIRD PROBLEM MENTIONED IN ORIGINAL POST
              //*          isr.close();
              //          is.close();
              //          os.close();
                        osw.close();
                        fis.close();
   /* */                sock.close();
                    } catch (UnknownHostException e) {
                        e.printStackTrace();
                    } catch (IOException e) {
                        e.printStackTrace();
                    }  finally{
                        try{
         //                   sock.close();
                        }
                        catch(Exception e){
                            e.printStackTrace();
                        }
                    } 
            	}
	          }).start();
	        }
	    });
        
    }       
    public static void connection(Socket s,  String serverIP, int port) {
        try {   
        	Log.v(qcd, " before connecting ****...");
        	s.connect(new InetSocketAddress(serverIP, port), 120000);
        	Log.v(qcd, " socket connection DONE!! ");
        } catch (UnknownHostException e) {
            e.printStackTrace();
            Log.v(qcd, " Unknown host..."+e);
        } catch (IOException e) {
            e.printStackTrace();
            Log.v(qcd, " Failed to connect...   "+e);
        }
    }
    public void onActivityResult(int requestCode, int resultCode, Intent data) {
        if (resultCode == RESULT_OK) {
            if (requestCode == SELECT_PICTURE) {
            	img.setImageURI(null);
                Uri selectedImageUri = data.getData();
                selectedImagePath = getPath(selectedImageUri);
                TextView path = (TextView) findViewById(R.id.tvPath);
                path.setText("Image Path : " + selectedImagePath);
                img.setImageURI(selectedImageUri);
                Log.v(qcd, selectedImagePath);
            }     
        }
    }

    public String getPath(Uri uri) {
        String[] projection = { MediaStore.Images.Media.DATA };
        Cursor cursor = managedQuery(uri, projection, null, null, null);
        int column_index = cursor
                .getColumnIndexOrThrow(MediaStore.Images.Media.DATA);
        cursor.moveToFirst();
        return cursor.getString(column_index);
    } 
}



Share this post


Link to post
Share on other sites


The do-while/read() code is below, what check (code) can I add to the code that would timeout the block, without causing an exception and without needing to restart the server.

You can use non-blocking IO. Java has another API for this, "NIO". Google for "java non-blocking server" for examples and explanations.

You may be able to get away with configuring the socket with a timeout, Google for "java socket SO_TIMEOUT".

 

I think the NIO API above was introduced to solve this problem without threading or setting timeouts on individuals sockets, though it is more complex again.

In a thread, the code is supposed to run sequentially. But a line not reach can cause a issue at an earlier line. Again this happened specifically with blocking. Not closing an object in a latter code line (not reached yet) can cause a block in an earlier read() line (more often in server code) 

Threads can have strange behaviour if you don't know what you are doing, in particular the order of operations visible across threads. Without going into too much detail, to make single threaded code fast, the compiler and CPU are allowed to do lots of surprising things, such as execute the code in a different order than you specified. Both are constrained in that the result of such an execution must not be distringuishable from what would have run if you executed each statement in the order you specified. This optimization breaks down if one thread looks at work that another thread is doing, it can observe these unexpected orderings of actions. Since nobody wants to lose the single threaded optimisations, the solution is that CPU and compilers have special features (memory barriers, sychronisation primitives) which allow you to control how threads view each other's actions.

Long story short, multi-threading is hard. I'm not sure this is 100% related to the issue you describe, but I think trying to mix it in when you're struggling with sockets seems a bad idea

My last post in the previous thread I mentioned all was now working fine. Well that was until I decided to send and receive jpeg file integers and text file with data continuously between client and server without closing either. It runs guaranteed. By the third cycle (sometimes second cycle) the application is forced stopped at the end of the run.

Which "Application", the client or the server? Is there no error message at all? What about the process exit status?

Can you reproduce this behaviour running in a debugger? If all else fails, you could add print statements to try help find the code that is running just before the crash.

Share this post


Link to post
Share on other sites

Just in general, I think you should look into extracting more helper functions in your code, ones that do a simple thing well.

Here is an example of what that might look like with your current code: 

public class FileServer {                 
 
    public static  void main(String[] args) throws IOException {
        ServerSocket server = new ServerSocket(57925);
        System.out.println("Server at " + InetAddress.getLocalHost().getHostAddress() + " is listening on port " + server.getLocalPort()); 
        while (true) {
            System.out.println("Waiting for connection...");
            try (Socket client = server.accept()) {
                System.out.println("Client at " + client);
                processConnection(client);
            } catch (IOException e) {
                System.out.println("An error occurred processing client " + client);
                e.printStackTrace();
            }
        } 
    }
 
    private static void processConnection(Socket clientSocket) throws IOException {
        byte [] clientData = readFromClient(clientSocket);
        writeToFile(someFilename, clietnData);
        byte [] serverData = anyOtherServerSideProcessing(clientData);
        writeToClient(clientSocket, data);
    }
 
    private static byte [] readFromClient(Socket clientSocket) throws IOException {
        try (InputStream fromClient = clientSocket.getInputStream()) {
            int size = readInteger(fromClient);
            return readBytes(fromClient, size);
        }
    }
 
    private static void writeToClient(Socket clientSocket, byte [] data) throws IOException {
        try (OutputStream toClient = clientSocket.getOutputStream()) {
            sendInteger(toClient, data.length);
            sendBytes(toClient, data);
        }
    }
 
    private static void writeToFile(String filename, byte [] data) throws IOException {
        try (FileOutputStream fos = new FileOutputStream(filename)) {
            fos.write(data, 0, data.length);
        }
    }
 
}
Edited by rip-off

Share this post


Link to post
Share on other sites

You can use non-blocking IO. Java has another API for this, "NIO". Google for "java non-blocking server" for examples and explanations. You may be able to get away with configuring the socket with a timeout, Google for "java socket SO_TIMEOUT".   I think the NIO API above was introduced to solve this problem without threading or setting timeouts on individuals sockets, though it is more complex again.

Is this similar to the second part of this SO answer? I saw it earlier on but didn't know how to integrate it into my code

 

Which "Application", the client or the server? Is there no error message at all?

Sorry i should have mentioned in original post. Its the client application that was forced stopped on the Android handset. Which is why I had concentrated my diagnosis on the client code  

 

 What about the process exit status? Can you reproduce this behaviour running in a debugger?

I couldn't reproduce in the debugger, though It runs to the end, frustratingly no exception dumb. Which makes it more difficult to know what the cause is

 

 If all else fails, you could add print statements to try help find the code that is running just before the crash.

Hmm... i did put lots of prints statements in the code, but its just occurring to me now I could improve that a little bit more, because on the second or third cycle of run the text file dumped to the client looses the data written to it.

[Just in case, i should mention that this is not quite the same previous as the problem in the previous thread. In the previous thread the data never gets written to the text file at all even in the first cycle]

 

Just in general, I think you should look into extracting more helper functions in your code, ones that do a simple thing well. Here is an example of what that might look like with your current code: 

 

Agreed ... better code structure, would even makes me see better. Starting on that now.  Many thanks 

Share this post


Link to post
Share on other sites
The traditional way to make sure sockets don't block the server is to use the select() call to figure out which sockets are available to read, and then call recv() (or recvfrom()) on each of those sockets. recv() is guaranteed to not block and return the available data if select() says it's readable. This is why recv() may return less than the full buffer you request -- it's guaranteed to not block if there is at least one byte!

The original Java sockets API didn't give very good access to these system calls, therefore you have to use the "nio" classes to build properly non-blocking servers in Java.

As far as "a line not reach can cause a issue at an earlier line" this is simply not true. (Well, for certain compute/memory workloads, the compiler or even CPU may in fact re-order operations, but I doubt this is your problem.)

If you run the same function in more than one thread, then one thread may get to line N+1 when the second thread is at line N.
If you run the code in a loop, then the second time through the loop may of course get to line N after the first time of the loop gets to line N+1.
If you believe line N+1 is not reached, then run your code in a debugger, put a breakpoint on line N+1, and see whether it is reached.

In general, though, based on the questions you're asking, I do not recommend you write network code at this point. There are too many other things you are not yet sure of, that get in the way of debugging the networking-specific bits. Networking, in itself, is hard, because it deals with distributed systems and very carefully constructed system call semantics. However, to use those calls correctly, you need to understand all of the underlying assumptions and conventions. It's pretty clear that you currently do not, and do not yet have the abiltity to isolate separate pieces of code to investigate and debug how they perform. I recommend you work on single-user programs for a bit longer until you learn all the necessary underlying skills and methods that will help you later understand networking. Based on my experience, this will actually be the faster path to understanding and success for you. Edited by hplus0603

Share this post


Link to post
Share on other sites

When I re-read my first post I see I hadn't read it throu a second time as my construction had been appalling, good you and ripoff could still manage to see the question. Have edited it properly however

In general, though, based on the questions you're asking, I do not recommend you write network code at this point. There are too many other things you are not yet sure of, that get in the way of debugging the networking-specific bits. Networking, in itself, is hard, because it deals with distributed systems and very carefully constructed system call semantics. However, to use those calls correctly, you need to understand all of the underlying assumptions and conventions. It's pretty clear that you currently do not, and do not yet have the abiltity to isolate separate pieces of code to investigate and debug how they perform. I recommend you work on single-user programs for a bit longer until you learn all the necessary underlying skills and methods that will help you later understand networking. Based on my experience, this will actually be the faster path to understanding and success for you.

This is tough to swallow though I know you must be correct. Its Sad to accept because I though I was almost there. But when I begin to read 'use NIO classes to create non-blocking server', I knew I was in deep shit as this equivalent to starting all over again. And as you mentioned I reckon it would even be way tougher than what I've experienced so far.  

This shock will be difficult to recover from but when I do (if I do at all) , I will have to re-design the whole game so as to drop to networking part of it. 

I'm just still surprised though that based on its practical advantage NIO tcp  examples don't flood the internet more than the simplistic and impractical type of examples that I ended up following

Sad but thanks for the info

Share this post


Link to post
Share on other sites

It's a tough pill to swallow, but we've all been there at some stage in our development careers I think.  Just use an existing library, and get good at coding most other things and from that experience you will gain enough knowledge to get through this barrier.

Share this post


Link to post
Share on other sites
Just use an existing library, and get good at coding most other things and from that experience you will gain enough knowledge to get through this barrier.

Could you suggest one - a proper free package with simple APIs and with a tutorial for beginners? 

Share this post


Link to post
Share on other sites

I agree that you need to nail down more general Java programming concepts before proceeding, and to work on code quality to make it easier to find and understand bugs. Here's a short code review to get you started on the sort of things you should consider:

  • You don't initialise `int bytesRead` but you initialise everything else. Be consistent (and ideally initialise everything).
  • You create an InputStreamReader around the InputStream, and then a BufferedReader around that, but then you go back to reading from the InputStream directly later. I'm pretty sure you were warned not to do this in a previous thread, because some of your data can get 'lost' inside the BufferedReader.
  • What happens if bytesRead is less than zero? What should happen?
  • Your client thread has a 'finally' block that does absolutely nothing. However exceptions thrown from that thread are probably what crashes your Android app.
  • Create separate functions for coherent blocks of functionality. For some reason your server code has all the reading and writing code mashed inline in one big loop, but you've factored out 'closeFile' when it's essentially a one-line operation. A more natural division here would be one function to read the data, one to write the file, one to read the file, one to send it. Then you don't end up with atrocities like 12 close() calls in a row, because you wouldn't have all of those objects in one place.
  • All of the above is doubly important when you're trying to wedge all that into an inline thread definition. It's almost impossible to debug code like that. Instead of `onClick` creating this anonymous class, you should probably move all that into a new class in a new file.
  • Clean it up. Put in consistent indenting, remove unused code, remove unused comments.

Share this post


Link to post
Share on other sites

 

I agree that you need to nail down more general Java programming concepts before proceeding, and to work on code quality to make it easier to find and understand bugs. Here's a short code review to get you started on the sort of things you should consider:

  • You don't initialise `int bytesRead` but you initialise everything else. Be consistent (and ideally initialise everything).
  • You create an InputStreamReader around the InputStream, and then a BufferedReader around that, but then you go back to reading from the InputStream directly later. I'm pretty sure you were warned not to do this in a previous thread, because some of your data can get 'lost' inside the BufferedReader.
  • What happens if bytesRead is less than zero? What should happen?
  • Your client thread has a 'finally' block that does absolutely nothing. However exceptions thrown from that thread are probably what crashes your Android app.
  • Create separate functions for coherent blocks of functionality. For some reason your server code has all the reading and writing code mashed inline in one big loop, but you've factored out 'closeFile' when it's essentially a one-line operation. A more natural division here would be one function to read the data, one to write the file, one to read the file, one to send it. Then you don't end up with atrocities like 12 close() calls in a row, because you wouldn't have all of those objects in one place.
  • All of the above is doubly important when you're trying to wedge all that into an inline thread definition. It's almost impossible to debug code like that. Instead of `onClick` creating this anonymous class, you should probably move all that into a new class in a new file.
  • Clean it up. Put in consistent indenting, remove unused code, remove unused comments.

Thanks. This restructuring will certainly help overall. Particularly in the new setup i'm thinking about (explained below)

After hplus advice my perspective and target is changing. As you will see from previous posts, my original aim for this particular thread was to make my code non-blocking (or block-proof). But since I've been told I've got to use NIO classes (which is even harder than what i've done already and worse still, to me- it's like starting all over again), I've been in a state in a state of reflection since (to put it mildly). I'm thinking now, that in the short term I will redesign my game to exclude network coding but instead setup a learning network coding framework on my Android devices 

For learning purposes I need to test some stuff exclusively on Android devices. To this end I've got some few questions:: 

1. Is it possible to use IP 127.0.0.1   to run a client->server loopback on the same android device (not on emulator)?

2. Could both Client and Server run in the same application on same device? ( asking because most devices at the moment can run only one app at time. And since client and server can't run as separate app but have to run in the same application)

3. Could port numbers be used with mobile devices, if so, do I have to retrieve local host and port number for a mobile device the same as is done for PC server? 

4. Unrelated to above three ques, and just for curiosity sake - Why is it that despite the practical advantage of NIO api for tcp,  their examples are not so common on the internet? In fact every time I every time I've searched I've ended up with the simplistic and impractical type of examples that I ended up following

Share this post


Link to post
Share on other sites

I'm not familiar with Android development, but typically processes on the same machine can talk to one another on the loopback address.

If I were you, I'd think about just writing some non-Android based networking applications to experiment and get familiar with the topic, adding Android and Emulators into the mix is only going to make things a little harder. Writing small console applications is a great way to learn, trying to do your learning by building your desired functionality in an already complex application is a recipe for lots of unrelated issues to slow you down.


Why is it that despite the practical advantage of NIO api for tcp,  their examples are not so common on the internet? In fact every time I every time I've searched I've ended up with the simplistic and impractical type of examples that I ended up following

Well blocking I/O still has a place for simple use cases. For example, if you want to make a quick web request as part of a larger program, then a non-blocking solution would actually be quite complicated - particularly if you want to use the result immediately. Just throwing a quick timeout on a blocking request can be acceptable.

The audience of examples on the Internet is people new to the subject of networking. Better to start with the basics, sockets, ports, hostnames, etc in the simpler environment of blocking I/O. Examples typically try to get to the core of what is being taught, and actually try to avoid being too large as it can distract from the subject at hand. One suspects that sometimes the people who write these examples are often just graduated from being beginners themselves.

Share this post


Link to post
Share on other sites

I'm not familiar with Android development, but typically processes on the same machine can talk to one another on the loopback address.

If I were you, I'd think about just writing some non-Android based networking applications to experiment and get familiar with the topic, adding Android and Emulators into the mix is only going to make things a little harder. Writing small console applications is a great way to learn, trying to do your learning by building your desired functionality in an already complex application is a recipe for lots of unrelated issues to slow you down.

Hmm... I see your point.

Yeah, I think that should be a better way to go

Well blocking I/O still has a place for simple use cases. For example, if you want to make a quick web request as part of a larger program, then a non-blocking solution would actually be quite complicated - particularly if you want to use the result immediately. Just throwing a quick timeout on a blocking request can be acceptable. The audience of examples on the Internet is people new to the subject of networking. Better to start with the basics, sockets, ports, hostnames, etc in the simpler environment of blocking I/O. Examples typically try to get to the core of what is being taught, and actually try to avoid being too large as it can distract from the subject at hand. One suspects that sometimes the people who write these examples are often just graduated from being beginners themselves.

Ah... get it 

thanks

Share this post


Link to post
Share on other sites

But since I've been told I've got to use NIO classes (which is even harder than what i've done already and worse still, to me- it's like starting all over again)


If your code is clean, this is not such a big change. After all, your file loading and saving should be completely separate.

However, it's standard to use regular blocking I/O for Android networking since you're usually only making one call at a time, in a background thread. Not so much so for servers, perhaps. But nobody claimed this was introductory-level material.

For learning purposes I need to test some stuff exclusively on Android devices.


No, really you want to test everything on a desktop PC where you have full control of the environment.

1. Is it possible to use IP 127.0.0.1   to run a client->server loopback on the same android device (not on emulator)?


Android devices are not designed to act as servers.

2. Could both Client and Server run in the same application on same device? ( asking because most devices at the moment can run only one app at time. And since client and server can't run as separate app but have to run in the same application)


Yes, but that's an order of magnitude more complex than doing what you've been trying so far.

3. Could port numbers be used with mobile devices


The whole internet works with port numbers. TCP/IP doesn't care whether a device fits in your pocket or not.

Share this post


Link to post
Share on other sites

If your code is clean, this is not such a big change. After all, your file loading and saving should be completely separate.

A big relieve to read this,  ... means I don't have to discard what i coded already to accommodate NIO api. So lately I started doing some digging on NIO, nothing solid yet...

 

You create an InputStreamReader around the InputStream, and then a BufferedReader around that, but then you go back to reading from the InputStream directly later. I'm pretty sure you were warned not to do this in a previous thread, because some of your data can get 'lost' inside the BufferedReader.

I was trying to restructure my code as you advised, but I see that i need BufferedReader instance to read a line of string-text. Because Client sends a string-text data first before sending the  file (image or text). i.e

          BufferedReader br = new BufferedReader(isr);
          String number = br.readLine();

I would have used InputStream only but on right-clicking on it and checking the doc I see no methods for reading a line of string/text. I know the string is still bytes but since it shouldn't  mix with the file bytes I'm thinking String number = br.readLine(); reads the string-text separately while InputStream is; is.read(...); Is then dedicated to reading the file bytes.

In essence it means not using BufferReaderObject.readLine() means I can't separately read the string text

I'm writing my understanding of this but like to be corrected so i can do it right. 

Share this post


Link to post
Share on other sites

I see that i need BufferedReader instance to read a line of string-text


Here's the problem: The other end may be sending "This is line 1\nThis is line 2\n" as two lines of text.
The underlying network may split that into the packets: "This ", "is line 1\nT", "his", " is line 2" and "\n"
(or any other way)
So, when you receive the first packet, you may or may not have a full string yet.
When you use a BufferedReader, and synchronous I/O, your program will wait for the full line to arrive in the thread that runs the BufferedReader code.
This is why a lot of Java code uses one thread per socket/client, and it performs poorly, AND it requires threading which causes bugs in programs that aren't 100% thread-safe.

What high-performance programs do is take over serialization themselves. They keep their own input buffer. Each time they get some data and extend the buffer, they check the buffer for whether it contains one (or more) full frames of data, and then extracts and processes those frames. (And then remove the parsed data from the head of the buffer -- a ring buffer is often useful.)
What enterprise applications that don't care about performance do, is use Java built-in serialization. However, that way lies network packets that are multiple kilobytes just to send the data "hello, world!" AND it's usually a big security hole, so I would strongly recommend against trying that route.

Share this post


Link to post
Share on other sites
What high-performance programs do is take over serialization themselves. They keep their own input buffer. Each time they get some data and extend the buffer, they check the buffer for whether it contains one (or more) full frames of data, and then extracts and processes those frames. (And then remove the parsed data from the head of the buffer -- a ring buffer is often useful.) What enterprise applications that don't care about performance do, is use Java built-in serialization. However, that way lies network packets that are multiple kilobytes just to send the data "hello, world!" AND it's usually a big security hole, so I would strongly recommend against trying that route.

 

I would like to follow your advice except that the second half of your post is a little bit too high level for me to translate back to "how to fix it", or better said I don't know how to construct the the fix. 

For instance as said in previous post, I'd like to use just InputStream as you and Kylotan advice, but how do I know how to read the first part of the received data separately form the second. (I'm avoiding the word "line", but still I need to separate the string data part from the file data stream part). Do I specify the exact number of bytes for the first dataset(string) and the second dataset (image file)? A little bit of pseudocode (2 ... 3 lines) will be highly appreciated. 

As a complete starter (in this branch of computing) I've always thrived when I see a little bit of snippet/APIs ex, otherwise I have no clue. 

Using only Inputstream instance, how do I avoid lumping the two data together? How do I see the end of the first part and the start of the second part?

Many thanks in advance

Share this post


Link to post
Share on other sites

how do I know how to read the first part of the received data separately form the second


When you call the native "receive()" function on the socket, it will return as much or little data as is available (but at least one byte.)
Thus, even if you ask for 1000 bytes, if there's only 3 bytes arrived, you will get 3 bytes.
The main problem with the original Java sockets is that you don't actually get access to that underlying native behavior, and thus old-school Java programs do terrible things like spawn a thread, which just asks for one byte at a time from a socket.
(As you can imagine, this generates garbage like crazy on the heap!)
Yes, it turns out, the creators of Java weren't super-humans who never made mistakes :-)

If you use java.net.Socket, and then call getInputStream(), then you may be able to check how much to ask for by calling available().
Thus, your "scan for input" would call available(), and then read that many bytes into your input buffer array.

However, it's better to use java.nio and tie a SocketChannel to the socket and set the SocketChannel in non-blocking mode.

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

Sign in to follow this