• Advertisement
Sign in to follow this  

problem with read() function in python

This topic is 3985 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

If you intended to correct an error in the post then please contact us.

Recommended Posts

I'm having a problem with using the read() function in python, it will read a file, but only the first couple hundred bytes. I can't find out why this is happening or how to fix it. After a lot of googling, I found it it may have something to do with blocking mode, but I don't know what that it is. I also checked my memory to make sure I had enough, which I do. The weirdest thing, is that i was working perfectly fine last night. So, how can I fix this?

Share this post


Link to post
Share on other sites
Advertisement
What are the parameters? Is it a binary file? If so, did you open it with "rb"? If it's not binary, and you're reading it as text, does it contain any weird characters (happens in some XML feeds).

Is there an error? Does the application hang? Crash? Exit? How many bytes do you read? Is it always the same number? If so, which character does it stop on?

Share this post


Link to post
Share on other sites
I haven't experienced this before, and I wasn't able to recreate it. I did a little searching, and I found this:

Quote:
from a thread on the Python-Dev mailing list:

The only safe way to read and write to the popen2 files and avoid blocking,
without special knowledge of the io behaviour of the executed command, is to
use non-blocking mode. To set a file object "f" into non-blocking mode
requires manipulating the file's file descriptor using the Python library
fnctl module as follows;

import os,fnctl

# get the file descriptor
fd = f.fileno()

# get the file's current flag settings
fl = fcntl.fcntl(fd, fcntl.F_GETFL)

# update the file's flags to put the file into non-blocking mode.
fcntl.fcntl(fd, fcntl.F_SETFL, fl | os.O_NONBLOCK)

Once a file is in non-blocking mode, the file object's read() and write()
methods cannot reliably be used. Instead you must use the os.read() and
os.write() methods on the fileno() of the file;

import os

str = os.read(f.fileno(), count)

count = os.write(f.fileno(), str)


It might be easier to just call file.read() until you've read all the data from the file, though. That would make your code easier to read, too.

- Mike

Share this post


Link to post
Share on other sites
@Antheus: its just a text file, and no it doesn't hang when I call the function. It just oputs text from the file like it should, except, instead of outputting the whole file, it only outputs the first 100-300 bytes. The amount it outputs changes from file to file.

@doctorsixstring: I'll try what you put in the quotes. As for calling file.read() multiple times, that doesn't work. The first file.read() reads the first 100-200 bytes, then the file position skips to the very end of the file. It seems to always stop on a null/whitespace byte, but its never the first null/whitespace byte in the file.

Edit: the fcntl module seems to onl be available in Unix, so that won't work.

Share this post


Link to post
Share on other sites
Could anyone else possibly help me? I tried reinstlling python, but tht didn't work.

Share this post


Link to post
Share on other sites
Its nothing complicted, its just reading from a file:

f = open("test.txt", "r")
f.read()

Share this post


Link to post
Share on other sites
Quote:
Original post by ForD
Its nothing complicted, its just reading from a file:

f = open("test.txt", "r")
f.read()

Show us more code. Like the buffer you read into and what you do with it, and whether you also try to write to the file, etc. Show us your whole program.

Share this post


Link to post
Share on other sites
Quote:
Original post by Oluseyi
Show us more code. Like the buffer you read into and what you do with it, and whether you also try to write to the file, etc. Show us your whole program.

That is all of my code. That's literally all I'm trying to do, open file and read it. Nothing more.

Share this post


Link to post
Share on other sites
Quote:
Original post by ForD
Quote:
Original post by Oluseyi
Show us more code. Like the buffer you read into and what you do with it, and whether you also try to write to the file, etc. Show us your whole program.

That is all of my code. That's literally all I'm trying to do, open file and read it. Nothing more.


If it doesn't appear to be "hanging", then it's not a problem with blocking.

I take it you are doing this at the interactive prompt (so that you can see the value of the expression 'x.read()')?

What kind of file is it? What are you expecting to see, and why? What does it look like if you open it in a text editor (such as Notepad)?

Share this post


Link to post
Share on other sites
Quote:
Original post by Zahlman
Quote:
Original post by ForD
Quote:
Original post by Oluseyi
Show us more code. Like the buffer you read into and what you do with it, and whether you also try to write to the file, etc. Show us your whole program.

That is all of my code. That's literally all I'm trying to do, open file and read it. Nothing more.


If it doesn't appear to be "hanging", then it's not a problem with blocking.

I take it you are doing this at the interactive prompt (so that you can see the value of the expression 'x.read()')?

What kind of file is it? What are you expecting to see, and why? What does it look like if you open it in a text editor (such as Notepad)?

Yes, I'm doing it through IDLE.

Its a plugin file for a game tht I've made a text file, so I'm expecting to see some data that tells the game to do certain things. If I open it up in notepad,I see this data, if I use the read() function, I only see the first prt of the data. But, like I said before, the read() function was working just fine a couple of nights ago.

Share this post


Link to post
Share on other sites
Quote:
Original post by ForD
That is all of my code. That's literally all I'm trying to do, open file and read it. Nothing more.

Quote:
Original post by ForD
Its a plugin file for a game tht I've made a text file, so I'm expecting to see some data that tells the game to do certain things. If I open it up in notepad,I see this data, if I use the read() function, I only see the first prt of the data. But, like I said before, the read() function was working just fine a couple of nights ago.

Interesting. When was the text file last modified? The first thing that comes to mind is that you've inadvertently introduced an EOF into the text file somewhere (though why Notepad shows it in entirety is a good question). Can you upload the text file and let us test a simple read from the interactive shell?

Share this post


Link to post
Share on other sites
Quote:
Original post by Oluseyi
Looks like a Unicode problem, specifically a codepage mismatch. What was the original encoding of the file?

Latin-1 I believe. What's a codepage mismatch?

Share this post


Link to post
Share on other sites
Quote:
Original post by Antheus
Did you try f = open("test.txt", "rb")

You are reading a binary file.

Thank you, that worked perfectly.

Share this post


Link to post
Share on other sites
Sign in to follow this  

  • Advertisement