Print Python output by PHP Code

Question

I have a scraper which scrape one site (Written in python). While scraping the site, that print lines which are about to write in CSV. Scraper has been written in Python and now I want to execute it via PHP code. My question is

how can I print each line which is getting printed by python code.

I have used exec function but it is none of my use and gives output after executing all the program. So;

Is it possible to get python output printed while it is getting executed via PHP.

Falsity

Answer 1

If i understand it well, your python scraper output to a file and you want to "live" display the output via php. What about doing and loop in php in which you use filemtime to know whether or not the file has been updated? You might add a little sleep in order not to overload your server.

If your are using a web page, you may use AJAX to reload only the concerned part of the page at a regular interval.

Hoping this helps you.

Fishy

Thanks, Seems like it will work. You got my point well. But is it possible to open single file twice with Python as well as in PHP at a same time. Well as far as I know, We can't do that. – Falsity

Python seems able to do it: #9510683 For PHP, since you should be using read-only mode, I don't think it will cause any trouble – Fishy

Indeed there is a small problem. Python seems to write the content only when file.close() is called. But PHP managed to open the file in read-only mode while python was using it (or so it seems). – Fishy

Instead of file.close() you can call file.flush() which updates the file content on the disk and leaves it open. – Supination

Answer 2

Simple case

Assuming execution of scraper is limited to php runtime, run it via popen: http://php.net/manual/en/function.popen.php

More involved case

If you want scraper to run on background and only connect to it vis php from time to time, you can either use some pub/sub toolkit or implement a small web server in the scraper that you can fetch result updates with fopen("https://localhost:port/...") or curl. Many other rpc mechanisms are possible, domain sockets, watching a file, sysv rpc...

Kamp

Answer 3

I'd communicate using stdout instead of a file. Meaning the python script writes to stdout and the php script reads that.

Using proc_open you can control the python process from php and also read it's output.

Antiperistalsis

I have checked with proc_open, but it is working same as exec. It take time to execute Python code. And once the execution got finished it execute rest of the PHP code. Is there any way to handle both process simultaneously? – Falsity

Answer 4

Instead of using exec you could use passthru, that will output the data directly to the browser. http://php.net/manual/en/function.passthru.php

That should be enough to get the println from your script.

Reciprocity

Answer 5

I think I have a fair idea of what you are saying put I am not too sure what you mean.

If you mean to say that everytime the python script does a print, you want the php code to output what was print?

If that is the case you could pass it as a POST DATA via HTTP. That is instead of printing in Python, you could send it to the PHP Script, which on receiving the data would print it.

I am not too sure if this is what you want though.

Fortunio

Answer 6

For proper communication you need to setup any medium, so you can use fifo, through it you can write string in python and read it with php.

For PHP fifo http://php.net/manual/en/function.posix-mkfifo.php

For Python http://www.doughellmann.com/PyMOTW/Queue/

Roberge

Answer 7

Simply use system() instead of exec(). exec() saves all lines of stdout output of the external program into an array, but system() flushes stdout output "as it happens".

Oxidase