Detailed explanation of newline parameter instance in Python open function

The origin of the problem

Specific examples

summary

I'm reading pythoncsv Module document See such a sentence

If csvfile It's the file object , Open it with newline=‘’.
Its remarks ： If not specified newline=‘’, The newline character embedded in quotation marks will not be correctly parsed , And when writing , Use \r\n The newline platform will have redundant \r write in . because csv The module will execute its own （ Universal ） Line break handling , Therefore, it is specified that newline=‘’ It should always be safe .

I was just thinking open Function newline Function of parameters , Because I was using open Function has never been set newline Parameters , Only from the above official remarks newline Parameter can help handle the problem of newline parsing

And the line feed characters of different operating systems are different ：

Unix End of line agreement ‘\n’、Windows The agreement of ‘\r\n’ And the old version Macintosh The agreement of ‘\r’

Breaking my original idea that the newline character is \n

python The official document is right newline Parameter interpretation ：

newline control universal newlines How the model works （ It applies only to text mode ）. It can be None,‘’,‘\n’,‘\r’ and ‘\r\n’. How it works :
When reading input from a stream , If newline by None, Then enable the general line feed mode . The lines in the input can be in the form of ‘\n’,‘\r’ or ‘\r\n’ ending , These lines are translated into ‘\n’ Before returning to the caller . If it is ‘’, Then enable the general line feed mode , But the end of the line will be returned to the caller untranslated . If it has any other legal value , Then the input line is terminated only by the given string , And the end of the line will be returned to the caller who did not call .
When writing output to stream , If newline by None, Then write any ‘\n’ All characters will be converted to the system default line separator os.linesep. If newline yes ‘’ or ‘\n’, No translation . If newline Is any other legal value , Then write any ‘\n’ The character will be converted to the given string .

From this, we can understand why we originally used open() Use it when writing \n It means that the end of the line will return... When the line breaks and the text file is read \n

No... Is specified when writing newline The parameter will \n Translated into the system default line separator (\r\n)

Read without specifying newline Parameter will split the line (\r\n) Translated into \n

Back to the above , Why are you reading and writing csv File should be set newline='' Well ？

pythoncsv Official documents explain this problem ( This introduces the second method to solve the line feed problem , I'll introduce it later )

Dialect.lineterminator
Put it in writer The end of the resulting line , The default is ‘\r\n’.
annotation reader Hard coded , identifies ‘\r’ or ‘\n’ As the end of the line , And ignore lineterminator. This behavior may change in the future .

In plain English writerow() Method When a row of data is written, the end of the row is It will be followed by a default line break (\r\n)( namely csv Yes, it will ’ A line of data \r\n’ Write to memory , At this time, this row of data is still in memory , The file has not been written yet ) After that, the execution code will actually write to the file according to different newline Parameters for translation
And in the direction of txt file Use write() Method When writing content, we add line breaks manually \n( The data in the memory is what we write , and Nothing else is implicitly added ) Then the execution code actually writes to the file according to newline Parameters for translation , This is the difference between the two
Specific process ：
newline=‘’
writer.writerow(‘line’) It is actually writing to memory ’line\r\n’ --》 Execute code , write file , according to newline=‘’, There will be no translation --》 File final write ’line\r\n’
newline=None( Default )
f.write(‘line\n’) Direct will ’line\n’ Write to memory --》 Execute code , write file , according to newline=None, take \n Translated into \r\n --》 File final write ’line\r\n’

case1: w newline=‘’ r newline=‘’

case2: w newline=‘\r’ r newline=‘’

case3: w newline=‘\r\n’ r newline=‘’

case4: w newline=None r newline=None

case5: File written as \r\r\n File read newline=‘\r’

Why the report is wrong ：

csv.reader How to read \r\r\n Of ： Encountered while reading \r Think the line is over , Once again \r It also thinks that a line ends ( Thus, an empty string list is returned ), encounter \n Can't explain –》 Report errors

case6: File written as \r\r\n File read newline=‘\n’

case7: File written as \r\r\n File read newline=‘\r\n’

case8: File written as \r\r File read newline=‘\r’

The second method ： By setting csv.writer Methods lineterminator Parameters

above-mentioned lineterminator Parameter control writer Write the implicit terminator following each line , The default is ’\r\n’, So we need to set lineterminator=‘\n’, There is no need to set when reading newline Parameter to get the desired effect

This is about python open Function newline This is the end of the article on detailed explanation of parameter examples , More about python open function newline Please search the previous articles of SDN or continue to browse the related articles below. I hope you will support SDN more in the future ！