Python not reading from file

I am trying to loop over the lines of a text file which is verifiably non-empty and I am running into problems with my script. In my attempt to debug what I wrote, I figured I would make sure my script is properly reading from the file, so I am currently trying to print every line in it.

At first I tried using the usual way of doing this in Python i.e.:

with open('file.txt') as fo:
for line in fo:
print line


but my script is not printing anything. I then tried storing all of the lines in a list like so:

with open('file.txt') as fo:
flines = fo.readlines()
print flines


and yet my program still outputs an empty list (i.e. ). I have also tried making sure that my file pointer is pointing to the beginning of the file using fo.seek(0) before attempting to read from it, yet that also does not work.

fo.seek(0)

I have spent some time reading solutions to similar questions posted on here, but so far nothing I have tried has worked. I do not know how such an elementary I/O operation is giving me so much trouble, but I must be missing something basic and would thus really appreciate any help/suggestions.

EDIT: Here is the part of my script which is causing the problem:

import subprocess as sbp
with open('conf_15000.xyz','w') as fo:
p1 =sbp.Popen(['head','-n', '9','nnp-pos-1.xyz'],stdout=sbp.PIPE)
p2 = sbp.Popen(['tail','-n', '3'],stdin=p1.stdout,stdout=fo)

with open('conf_15000.xyz','r') as fp:
fp.seek(0)
flines = fp.readlines()

print flines


And here is an exerpt from the nnp-pos-1.xyz file (all lines have the same format and there are 370642 of them in total):

nnp-pos-1.xyz

Ti 32.9136715924 28.5387609200 24.6554922872
O 39.9997000300 35.1489480846 22.8396092714
O 33.7314699265 30.3398473499 23.8866085372
Ti 27.7756767925 31.3455930970 25.9779887743
O 31.1520937719 29.0752315770 25.4786577758
O 26.1870965535 32.4876155555 26.3346205619
Ti 38.4478275543 25.5734609650 22.0654953429
O 24.1328940232 31.3858060129 28.8575469919
O 38.6506317714 27.3779871011 22.6552032123
Ti 40.5617501289 27.5095900385 22.8436684314
O 38.2400600469 29.1828342919 20.7853056680
O 38.8481088254 27.2704154737 26.9590081202


When running the script, the file being read from (conf_15000.xyz) gets written to properly, however I cannot seem to be able to read from it at runtime.

conf_15000.xyz

EDIT-2: Following sudonym's recommendation I am using the absolute file path and am checking whether or not the file is empty before reading from it by adding the following unindented lines between the two with statements I wrote in my previous edit:

with

print os.path.isfile(r'full/path/to/file')
print (os.stat(r'full/path/to/file').st_size != 0)


The first boolean evaluates to True (meaning the file exists) while the second evaluates to False (meaning the file is empty). This is very strange because both of these lines are added after I close the file pointer fo which writes to the file and also because the file being written to (and subsequently read from with fp) is not empty after I execute the script (in fact, it contains all the lines it is supposed to).

True

False

fo

fp

Please make sure that the code included in the question is self-contained enough that someone can copy-and-paste without modifications to see the problem themselves. If that means including code that writes to file.txt so we can be certain it isn't empty, for example, doing that in the content included in the question itself helps the "verifiable" element of the Minimal, Complete, and Verifiable example definition be met.
– Charles Duffy
Jun 29 at 21:07

file.txt

I recommend trying the absolute path to file.txt. Depending on where the file is and where you are running the script from, it might be opening a new file that is empty. Note that this is intended to verify that you are opening exactly the file.txt you think you are, and not as the permanent solution.
– MoxieBall
Jun 29 at 21:08

file.txt

file.txt

...so, does the issue still reproduce if you run f = open('file.txt', 'w'); f.write('line onenline twon'); f.close() immediately before the code here? (Because it's creating the file with the same relative path used to open for read, that approach moots the concern MoxieBall raises in the other comment)
– Charles Duffy
Jun 29 at 21:11

f = open('file.txt', 'w'); f.write('line onenline twon'); f.close()

Hi, thank you for the prompt answers. I'm currently editing my question to make it more self-contained and to clarify certain things (I am indeed writing to the file in the same script as I am reading from it which is probably contributing to the bug).
– Nicolas Gastellu
Jun 29 at 21:15

Could you modify the code to where the issue can reproduce with only the exact nnp-pos-1.xyz file you include here? (Obviously can't do that when you seek thousands of lines in before starting reading, when the sample given is only 12 lines long). The goal, again, is to let someone reproduce the problem on their own machine using only content directly copy-and-pasted from the question.
– Charles Duffy
Jun 29 at 22:20

nnp-pos-1.xyz

2 Answers
2

first, include the absolute path.
Second, check if the file actually exists and is not empty:

import os

FILEPATH = r'pathtofile.txt' # full path as raw string
if os.path.isfile(FILEPATH) and (os.stat(FILEPATH).st_size != 0):
with open(FILEPATH) as fo:
flines = fo.readlines()
print flines
else:
print FILEPATH, "doesn't exist or is empty"


I have tried using the absolute path in the way you just suggested and the result is left unchanged. The script also correctly produces the file; I can cat it and verify that all the lines that need to be there have been written using wc -l. Thank you very much for your suggestion though!
– Nicolas Gastellu
Jun 29 at 21:34

cat

wc -l

are you expecting content with any non-standard encoding?
– sudonym
Jun 29 at 21:39

Actually, I just realised that you are right, the file is apparently empty when I want to read from it (even though I try reading from it after writing to it and closing the relevant file pointer); (os.stat(FILEPATH).st_size != 0) evaluates to False for some odd reason...
– Nicolas Gastellu
Jun 29 at 21:45

(os.stat(FILEPATH).st_size != 0)

False

No, the encoding is standard (AFAIK); the file contains only numbers and the letters Ti and O.
– Nicolas Gastellu
Jun 29 at 21:46

Ti

O

ok - is your issue resolved?
– sudonym
Jun 29 at 21:47

You probably need to flush() the buffers first (and maybe call os.fsync() too) - after writing and before reading.

flush()

os.fsync()

See file.flush() and this post.

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.