What are the best practices around creating flat file database structures in PHP?
A lot of more matured PHP flat file frameworks out there which I attempt to implement SQL-like query syntax which is over the top for my purposes in most cases. (I would just use a database at that point).
Are there any elegant tricks out there to get good performance and features with a small code overhead?
Well, what is the nature of the flat databases. Are they large or small. Is it simple arrays with arrays in them? if its something simple say userprofiles built as such:
$user = array("name" => "bob",
"age" => 20,
"websites" => array("example.com","bob.example.com","bob2.example.com"),
"and_one" => "more");
and to save or update the db record for that user.
$dir = "../userdata/"; //make sure to put it bellow what the server can reach.
file_put_contents($dir.$user['name'],serialize($user));
and to load the record for the user
function &get_user($name){
return unserialize(file_get_contents("../userdata/".$name));
}
but again this implementation will vary on the application and nature of the database you need.
You might consider SQLite. It's almost as simple as flat files, but you do get a SQL engine for querying. It works well with PHP too.
In my opinion, using a "Flat File Database" in the sense you're meaning (and the answer you've accepted) isn't necessarily the best way to go about things. First of all, using serialize() and unserialize() can cause MAJOR headaches if someone gets in and edits the file (they can, in fact, put arbitrary code in your "database" to be run each time.)
Personally, I'd say - why not look to the future? There have been so many times that I've had issues because I've been creating my own "proprietary" files, and the project has exploded to a point where it needs a database, and I'm thinking "you know, I wish I'd written this for a database to start with" - because the refactoring of the code takes way too much time and effort.
From this I've learnt that future proofing my application so that when it gets bigger I don't have to go and spend days refactoring is the way to go forward. How do I do this?
SQLite. It works as a database, uses SQL, and is pretty easy to change over to MySQL (especially if you're using abstracted classes for database manipulation like I do!)
In fact, especially with the "accepted answer"'s method, it can drastically cut the memory usage of your app (you don't have to load all the "RECORDS" into PHP)
serialize() can be pretty useful for that as well. I think the trick to coming up with a viable system is finding some way to index the data nodes without killing yourself with complexity. –
Swaim One framework I'm considering would be for a blogging platform. Since just about any possible view of data you would want would be sorted by date, I was thinking about this structure:
One directory per content node:
./content/YYYYMMDDHHMMSS/
Subdirectories of each node including
/tags
/authors
/comments
As well as simple text files in the node directory for pre- and post-rendered content and the like.
This would allow a simple PHP glob() call (and probably a reversal of the result array) to query on just about anything within the content structure:
glob("content/*/tags/funny");
Would return paths including all articles tagged "funny".
Here's the code we use for Lilina:
<?php
/**
* Handler for persistent data files
*
* @author Ryan McCue <[email protected]>
* @package Lilina
* @version 1.0
* @license http://opensource.org/licenses/gpl-license.php GNU Public License
*/
/**
* Handler for persistent data files
*
* @package Lilina
*/
class DataHandler {
/**
* Directory to store data.
*
* @since 1.0
*
* @var string
*/
protected $directory;
/**
* Constructor, duh.
*
* @since 1.0
* @uses $directory Holds the data directory, which the constructor sets.
*
* @param string $directory
*/
public function __construct($directory = null) {
if ($directory === null)
$directory = get_data_dir();
if (substr($directory, -1) != '/')
$directory .= '/';
$this->directory = (string) $directory;
}
/**
* Prepares filename and content for saving
*
* @since 1.0
* @uses $directory
* @uses put()
*
* @param string $filename Filename to save to
* @param string $content Content to save to cache
*/
public function save($filename, $content) {
$file = $this->directory . $filename;
if(!$this->put($file, $content)) {
trigger_error(get_class($this) . " error: Couldn't write to $file", E_USER_WARNING);
return false;
}
return true;
}
/**
* Saves data to file
*
* @since 1.0
* @uses $directory
*
* @param string $file Filename to save to
* @param string $data Data to save into $file
*/
protected function put($file, $data, $mode = false) {
if(file_exists($file) && file_get_contents($file) === $data) {
touch($file);
return true;
}
if(!$fp = @fopen($file, 'wb')) {
return false;
}
fwrite($fp, $data);
fclose($fp);
$this->chmod($file, $mode);
return true;
}
/**
* Change the file permissions
*
* @since 1.0
*
* @param string $file Absolute path to file
* @param integer $mode Octal mode
*/
protected function chmod($file, $mode = false){
if(!$mode)
$mode = 0644;
return @chmod($file, $mode);
}
/**
* Returns the content of the cached file if it is still valid
*
* @since 1.0
* @uses $directory
* @uses check() Check if cache file is still valid
*
* @param string $id Unique ID for content type, used to distinguish between different caches
* @return null|string Content of the cached file if valid, otherwise null
*/
public function load($filename) {
return $this->get($this->directory . $filename);
}
/**
* Returns the content of the file
*
* @since 1.0
* @uses $directory
* @uses check() Check if file is valid
*
* @param string $id Filename to load data from
* @return bool|string Content of the file if valid, otherwise null
*/
protected function get($filename) {
if(!$this->check($filename))
return null;
return file_get_contents($filename);
}
/**
* Check a file for validity
*
* Basically just a fancy alias for file_exists(), made primarily to be
* overriden.
*
* @since 1.0
* @uses $directory
*
* @param string $id Unique ID for content type, used to distinguish between different caches
* @return bool False if the cache doesn't exist or is invalid, otherwise true
*/
protected function check($filename){
return file_exists($filename);
}
/**
* Delete a file
*
* @param string $filename Unique ID
*/
public function delete($filename) {
return unlink($this->directory . $filename);
}
}
?>
It stores each entry as a separate file, which we found is efficient enough for use (no unneeded data is loaded and it's faster to save).
IMHO, you have two... er, three options if you want to avoid homebrewing something:
If you're familiar with PDO, you can install a PDO driver that supports SQLite. Never used it, but I have used PDO a ton with MySQL. I'm going to give this a shot on a current project.
Done this many times for relatively small amounts of data. XMLReader is a lightweight, read-forward, cursor-style class. SimpleXML makes it simple to read an XML document into an object that you can access just like any other class instance.
Good option for smallish amounts of data, just read/write file and json_decode/json_encode. Not sure if PHP offers a structure to navigate a JSON tree without loading it all in memory though.
If you're going to use a flat file to persist data, use XML to structure the data. PHP has a built-in XML parser.
If you want a human-readable result, you can also use this type of file :
ofaurax|27|male|something|
another|24|unknown||
...
This way, you have only one file, you can debug it (and manually fix) easily, you can add fields later (at the end of each line) and the PHP code is simple (for each line, split according to |).
However, the drawbacks is that you should parse the entire file to search something (if you have millions of entry, it's not fine) and you should handle the separator in data (for example if the nick is WaR|ordz).
I have written two simple functions designed to store data in a file. You can judge for yourself if it's useful in this case. The point is to save a php variable (if it's either an array a string or an object) to a file.
<?php
function varname(&$var) {
$oldvalue=$var;
$var='AAAAB3NzaC1yc2EAAAABIwAAAQEAqytmUAQKMOj24lAjqKJC2Gyqhbhb+DmB9eDDb8+QcFI+QOySUpYDn884rgKB6EAtoFyOZVMA6HlNj0VxMKAGE+sLTJ40rLTcieGRCeHJ/TI37e66OrjxgB+7tngKdvoG5EF9hnoGc4eTMpVUDdpAK3ykqR1FIclgk0whV7cEn/6K4697zgwwb5R2yva/zuTX+xKRqcZvyaF3Ur0Q8T+gvrAX8ktmpE18MjnA5JuGuZFZGFzQbvzCVdN52nu8i003GEFmzp0Ny57pWClKkAy3Q5P5AR2BCUwk8V0iEX3iu7J+b9pv4LRZBQkDujaAtSiAaeG2cjfzL9xIgWPf+J05IQ==';
foreach($GLOBALS as $var_name => $value) {
if ($value === 'AAAAB3NzaC1yc2EAAAABIwAAAQEAqytmUAQKMOj24lAjqKJC2Gyqhbhb+DmB9eDDb8+QcFI+QOySUpYDn884rgKB6EAtoFyOZVMA6HlNj0VxMKAGE+sLTJ40rLTcieGRCeHJ/TI37e66OrjxgB+7tngKdvoG5EF9hnoGc4eTMpVUDdpAK3ykqR1FIclgk0whV7cEn/6K4697zgwwb5R2yva/zuTX+xKRqcZvyaF3Ur0Q8T+gvrAX8ktmpE18MjnA5JuGuZFZGFzQbvzCVdN52nu8i003GEFmzp0Ny57pWClKkAy3Q5P5AR2BCUwk8V0iEX3iu7J+b9pv4LRZBQkDujaAtSiAaeG2cjfzL9xIgWPf+J05IQ==')
{
$var=$oldvalue;
return $var_name;
}
}
$var=$oldvalue;
return false;
}
function putphp(&$var, $file=false)
{
$varname=varname($var);
if(!$file)
{
$file=$varname.'.php';
}
$pathinfo=pathinfo($file);
if(file_exists($file))
{
if(is_dir($file))
{
$file=$pathinfo['dirname'].'/'.$pathinfo['basename'].'/'.$varname.'.php';
}
}
file_put_contents($file,'<?php'."\n\$".$varname.'='.var_export($var, true).";\n");
return true;
}
This one is inspiring as a practical solution:
https://github.com/mhgolkar/FlatFire
It uses multiple strategies to handling data...
[Copied from Readme File]
- STRUCTURED
Regular (table, row, column) format.
[DATABASE]
/ \
TX TableY
\_____________________________
|ROW_0 Colum_0 Colum_1 Colum_2|
|ROW_1 Colum_0 Colum_1 Colum_2|
|_____________________________|
- FREE
More creative data storing. You can store data in any structure you want for each (free) element, its similar to storing an array with a unique "Id".
[DATABASE]
/ \
EX ElementY (ID)
\________________
|Field_0 Value_0 |
|Field_1 Value_1 |
|Field_2 Value_2 |
|________________|
recall [ID]: get_free("ElementY") --> array([Field_0]=>Value_0,[Field_1]=>Value_1...
- MIXD (Mixed)
Mixed databases can store both free elements and tables.If you add a table to a free db or a free element to a structured db, flat fire will automatically convert FREE or SRCT to MIXD database.
[DATABASE]
/ \
EX TY
Just pointing out a potential problem with a flat file database with this type of system:
data|some text|more data
row 2 data|bla hbalh|more data
...etc
The problem is that the cell data contains a "|" or a "\n" then the data will be lost. Sometimes it would be easier to split by combinations of letters that most people wouldn't use.
For example:
Column splitter: #$% (Shift+345)
Row splitter: ^&* (Shift+678)
Text file: test data#$%blah blah#$%^&*new row#$%new row data 2
Then use: explode("#$%", $data); use foreach, the explode again to separate columns
Or anything along these lines. Also, I might add that flat file databases are good for systems with small amounts of data (ie. less than 20 rows), but become huge memory hogs for larger databases.
© 2022 - 2024 — McMap. All rights reserved.
for loop(unless I am using all the data in the folder), therefore it performs a lot faster than a database would. I would go into detail and give a very good answer but unfortunately this question is closed. – Dowling