前幾天跟同事討論如何在一次login php調用中保證error_log輸出的日志都有唯一的標識頭,結論是玩家帳號+當前時間+隨機數,在我們當前用戶量級條件下應該是滿足需求的,當然,這並不是本文討論的重點。
在確定這個簡單的方案之後,我在想兩個問題,也是今天本文討論的重點:
1.error_log調用是否可以保證輸出內容是完整的?
2.如果是完整的,那麼是怎樣保證的?
求證開始了,起初以為error_log調用中會對目標文件加文件鎖,直接看源碼(php5.6.12):
from: basic_functions.c
PHPAPI int _php_error_log_ex(int opt_err, char *message, int message_len, char *opt, char *headers TSRMLS_DC) /* {{{ */ { php_stream *stream = NULL; switch (opt_err) { case 1: /*send an email */ if (!php_mail(opt, PHP error_log message, message, headers, NULL TSRMLS_CC)) { return FAILURE; } break; case 2: /*send to an address */ php_error_docref(NULL TSRMLS_CC, E_WARNING, TCP/IP option not available!); return FAILURE; break; case 3: /*save to a file */ stream = php_stream_open_wrapper(opt, a, IGNORE_URL_WIN | REPORT_ERRORS, NULL); if (!stream) { return FAILURE; } php_stream_write(stream, message, message_len); php_stream_close(stream); break; case 4: /* send to SAPI */ if (sapi_module.log_message) { sapi_module.log_message(message TSRMLS_CC); } else { return FAILURE; } break; default: php_log_err(message TSRMLS_CC); break; } return SUCCESS; } /* }}} */
/* Writes a buffer directly to a stream, using multiple of the chunk size */ static size_t _php_stream_write_buffer(php_stream *stream, const char *buf, size_t count TSRMLS_DC) { size_t didwrite = 0, towrite, justwrote; /* if we have a seekable stream we need to ensure that data is written at the * current stream->position. This means invalidating the read buffer and then * performing a low-level seek */ if (stream->ops->seek && (stream->flags & PHP_STREAM_FLAG_NO_SEEK) == 0 && stream->readpos != stream->writepos) { stream->readpos = stream->writepos = 0; stream->ops->seek(stream, stream->position, SEEK_SET, &stream->position TSRMLS_CC); } while (count > 0) { towrite = count; if (towrite > stream->chunk_size) towrite = stream->chunk_size; justwrote = stream->ops->write(stream, buf, towrite TSRMLS_CC); /* convert justwrote to an integer, since normally it is unsigned */ if ((int)justwrote > 0) { buf += justwrote; count -= justwrote; didwrite += justwrote; /* Only screw with the buffer if we can seek, otherwise we lose data * buffered from fifos and sockets */ if (stream->ops->seek && (stream->flags & PHP_STREAM_FLAG_NO_SEEK) == 0) { stream->position += justwrote; } } else { break; } } return didwrite; }
多次write不能保證原子操作,那麼單次呢?
1.from:man write
If the file was open(2)ed with O_APPEND, the file offset is first set to the end of the file before writing. The adjustment of the file offset and the write operation are performed as an atomic step.
2.from:man write
Atomic/non-atomic: A write is atomic if the whole amount written in one operation is not interleaved with data from any other process. This is useful when there are multiple writers sending data to a single reader. Applications need to know how large a write request can be expected to be performed atomically. This maximum is called {PIPE_BUF}. This volume of IEEE Std 1003.1-2001 does not say whether write requests for more than {PIPE_BUF} bytes are atomic, but requires that writes of {PIPE_BUF} or fewer bytes shall be atomic.
man說的大概意思就是如果是O_APPEND模式,文件尾的定位與write調用是原子操作,不會存在write時文件尾還需要再調整,導致錯位。當前這個原子性,只保證open和之後的第一次write,後續的write就不保證了。像error_log那樣如果buf過長導致了多次write肯定就不保證buf一次完整輸出。
繼續深究,那麼單一次write是原子的嗎?man也給出了答案,不是的,只有要寫出的數據小於等於PIPE_BUF時才保證原子操作。
問題得到了理論上的解答,下面開始實驗驗證:
test_error_log.php
check_line.py
filename = ./append.txt for line in open(filename): print len(line)test.sh
#!/bin/bash rm -f append.txt for ((counter=0; counter < 10; ++counter)) do php test_error_log.php $counter $1 $2 & done sleep 2 #python check_line.py > a.txt python check_line.py | sort | uniq -c