24

I'm storing a list of items in a serialized array within a field in my database (I'm using PHP/MySQL).

I want to have a query that will select all the records that contain a specific one of these items that is in the array.

Something like this:

select * from table WHERE (an item in my array) = '$n'
5
  • 5
    Unless your items' have a very unique way of identifying them you'll probably be better off to store the serialized data as a table or something else. Commented Nov 7, 2010 at 3:57
  • 1
    which kind of array? integers? strings? mixed? Can you provide some sample records? Commented Nov 7, 2010 at 4:05
  • 1
    Never store serialized data that you want to search... Commented Nov 7, 2010 at 4:30
  • 1
    Select * from table where table_field like '%"enter_your_value"%' Commented Oct 4, 2015 at 12:30
  • Are you sure this problem is in any way related to PHP? Also, can you share sample data that helps others to understand the problem better? Commented Oct 15, 2024 at 12:31

14 Answers 14

26

As GWW says in the comments, if you need to query things this way, you really ought to be considering storing this data as something other than a big-ole-string (which is what your serialized array is).

If that's not possible (or you're just lazy), you can use the fact that the serialized array is just a big-ole-string, and figure out a LIKE clause to find matching records. The way PHP serializes data is pretty easy to figure out (hint: those numbers indicate lengths of things).

Now, if your serialized array is fairly complex, this will break down fast. But if it's a flat array, you should be able to do it.

Of course, you'll be using LIKE '%...%', so you'll get no help from any indicies, and performance will be very poor.

Which is why folks are suggesting you store that data in some normalized fashion, if you need to query "inside" it.

Sign up to request clarification or add additional context in comments.

Comments

22

If you have control of the data model, stuffing serialized data in the database will bite you in the long run just about always. However, oftentimes one does not have control over the data model, for example when working with certain open source content management systems. Drupal sticks a lot of serialized data in dumpster columns in lieu of a proper model. For example, ubercart has a 'data' column for all of its orders. Contributed modules need to attach data to the main order entity, so out of convenience they tack it onto the serialized blob. As a third party to this, I still need a way to get at some of the data stuffed in there to answer some questions.

a:4:{s:7:"cc_data";s:112:"6"CrIPY2IsMS1?blpMkwRj[XwCosb]gl<Dw_L(,Tq[xE)~(!$C"9Wn]bKYlAnS{[Kv[&Cq$xN-Jkr1qq<z](td]ve+{Xi!G0x:.O-"=yy*2KP0@z";s:7:"cc_txns";a:1:{s:10:"references";a:1:{i:0;a:2:{s:4:"card";s:4:"3092";s:7:"created";i:1296325512;}}}s:13:"recurring_fee";b:1;s:12:"old_order_id";s:2:"25";}

see that 'old_order_id'? thats the key I need to find out where this recurring order came from, but since not everybody uses the recurring orders module, there isnt a proper place to store it in the database, so the module developer opted to stuff it in that dumpster table.

My solution is to use a few targeted SUBSTRING_INDEX's to chisel off insignificant data until I've sculpted the resultant string into the data gemstone of my desires. Then I tack on a HAVING clause to find all that match, like so:

SELECT uo.*,
SUBSTRING_INDEX(
 SUBSTRING_INDEX(
  SUBSTRING_INDEX( uo.data, 'old_order_id' , -1 ),
 '";}', 1),
'"',-1) 
AS `old order id`
FROM `uc_orders AS `uo`
HAVING `old order id` = 25

The innermost SUBSTRING_INDEX gives me everything past the old_order_id, and the outer two clean up the remainder.

This complicated hackery is not something you want in code that runs more than once, more of a tool to get the data out of a table without having to resort to writing a php script.

Note that this could be simplified to merely

SELECT uo.*,
SUBSTRING_INDEX(
  SUBSTRING_INDEX( uo.data, '";}' , 1 ),
'"',-1) 
AS `old order id`
FROM `uc_orders` AS `uo`
HAVING `old order id` = 25

but that would only work in this specific case (the value I want is at the end of the data blob)

3 Comments

> "However, oftentimes one does not have control over the data model, for example when working with certain open source content management systems." That right there is the money quote for all of the others here who are admonishing people for not using the date in relational form. You can't fix the sins of others!
This is much more helpful than previous comments...like you said, Drupal/Ubercart are guilty of this, so telling people they should rebuild the house when all they are asking for is how to change a lightbulb is not entirely helpful
@MattKing telling people they should rebuild the house when all they are asking for is how to change a lightbulb +1 for great analogy!
18

So you mean to use MySQL to search in a PHP array that has been serialized with the serialize command and stored in a database field? My first reaction would be: OMG. My second reaction would be: why? The sensible thing to do is either:

  1. Retrieve the array into PHP, unserialize it and search in it
  2. Forget about storing the data in MySQL as serialized and store it as a regular table and index it for fast search

I would choose the second option, but I don't know your context.

Of course, if you'd really want to, you could try something with SUBSTRING or another MySQL function and try to manipulate the field, but I don't see why you'd want to. It's cumbersome, and it would be an unnecessary ugly hack. On the other hand, it's a puzzle, and people here tend to like puzzles, so if you really want to then post the contents of your field and we can give it a shot.

5 Comments

Yeah it looks as though my best option is to forget about the array and store it in its own table. Thanks!
> "OMG. My second reaction would be: why? The sensible thing to do is either:" How condescending. Often times, when working with CMS systems like Drupal and WordPress you have no control over the format that some other mindless programmer stored their data in.
The OMG comment has some merit. Writing a serialized code into database tables is problematic where the array is a serialized array of database keys. This SO question is a good example of the issue. Serializing an array of database keys, and storing them in the database, that will incur a performance penalty later for some kinds of lookup.
Though I agree with the fact that the data should not be stored in this mannter, as @MikeSchinkel mentioned, we sometimes don't have the option to do it differently.
It could be that youd search for a group of 'ids' storeed in an array/serialized text field in the dbase so as to see if an 'id' value had been used in a past transaction. Agreed a new table would work unless the OP was developing for Wordpress in which this would be considered metadata and need to go in the post meta table and done to avoid tons of new rows.
11

You can do it like this:

SELECT * FROM table_name WHERE some_field REGEXP '.*"item_key";s:[0-9]+:"item_value".*'

But anyway you should consider storing that data in a separate table.

3 Comments

Works! Did not realize you could do regex searches in mysql. Obviously very slow, but great ! Note that serialize does not escape special character values, so the item_value needs to be replaced with a db escaped only string.
@FlorianMertens Intereresting - What does this mean: db escaped only string?
It means that item_value in the above query should be escaped first. The escaping encoding being the database context itself. Otherwise, Item_value, if supplied by the user, may be used to hack the query and execute malicious queries.
5

Working with php serialized data is obviously quite ugly, but I've got this one liner mix of MySQL functions that help to sort that out:

select REPLACE(SUBSTRING_INDEX(SUBSTRING_INDEX(SUBSTRING_INDEX(searchColumn, 'fieldNameToExtract', -1), ';', 2), ':', -1), '"', '') AS extractedFieldName
from tableName as t 
having extractedFieldName = 'expressionFilter';

3 Comments

This is really helpful, thank you; I am faced with migrating a Wordpress site to Craft and a lot of important data is embedded in serialized arrays in the wp_postmeta table.
This was a plug and play in my case! Thanks!
This is interesting, but that big ole "REPLACE" looks dangerous to a novice ... can you give some details and explain more about how this works, what the user needs to replace?
4

How about you serialize the value you're searching for?

$sql = sprintf("select * from tbl WHERE serialized_col like  '%%%s%%'", serialize($n));

or

$sql = sprintf("select * from tbl WHERE serialized_col like  '%s%s%s'", '%', serialize($n), '%');

1 Comment

+1 for leading to a better way to get the escaped value into place, and for those that may think this protects them from sql injection remember you must still escape your value or the devil will get you. For example, sprintf("select * from tbl WHERE serialized_col like '%%%s%%'", $mysqli->real_escape_string(serialize($n)));
3

Simply use the IN statement, but put the field itself as array! Example:

SELECT id, title, page FROM pages WHERE 2 IN (child_of)

~ where '2' is the value i'm looking for inside the field 'child_of' that is a serialized array.

This serialized array was necessary because I cannot duplicate the records just for storing what id they were children of.

1 Comment

That will work if the value you're searching for is unlikely to exist anywhere else on accident. e.g. '2' would be a terrible thing to search for in a serialized array as php puts the size of its data values into the output string, thus you'd match many, many results. i.e. In my case I was searching for 25. It hit many false matches.
2

If I have attribute_dump field in log table and the value in one of its row has

a:69:{s:9:"status_id";s:1:"2";s:2:"id";s:5:"10215"}

If I want to fetch all rows having status_id is equal to 2, then the query would be

SELECT * FROM log WHERE attribute_dump REGEXP '.*"status_id";s:[0-9]+:"2".*'

Comments

2

There is a good REGEX answer above, but it assumes a key and value implementation. If you just have values in your serialized array, this worked for me:

value only

SELECT * FROM table WHERE your_field_here REGEXP '.*;s:[0-9]+:"your_value_here".*'

key and value

SELECT * FROM table WHERE your_field_here REGEXP '.*"array_key_here";s:[0-9]+:"your_value_here".*'

Comments

0

I was looking for the same and found that this query gives me the desired result. Although, I used Ruby on Rails to serialize the column product_ids

SELECT * FROM coupons WHERE product_ids LIKE '%---\\n- \\'71591\\'\\n%'

Comments

-1
Select * from table where table_field like '%"enter_your_value"%'

1 Comment

Lot of bad answers here but this is the best way to do it as far as I can tell. Because serialized arrays put quotes around stored values, this will make sure that even if you are looking for something like the number 1, you won't get a false positive from metadata on something like a:2:{i:0;s:1:"2";i:1;s:1:"7";} because s:1 is not in quotes. This is also much faster than resolving the filter in your PHP and more veristial than trying to pick out a substring based on location as the accepted answer suggests, .
-2

You may be looking for an SQL IN statement.

http://www.w3schools.com/sql/sql_in.asp

You'll have to break your array out a bit first, though. You can't just hand an array off to MySQL and expect it will know what to do with it. For that, you may try serializing it out with PHP's explode.

http://php.net/manual/en/function.explode.php

Comments

-2
select * from postmeta where meta_key = 'your_key'  and meta_value REGEXP  ('6')

Comments

-2
foreach( $result as $value ) {
   $hour = unserialize( $value->meta_value );
   if( $hour['date'] < $data['from'] ) {
     $sum = $sum + $hour['hours'];
   }
 }

1 Comment

While this code snippet may be the solution, including an explanation really helps to improve the quality of your post. Remember that you are answering the question for readers in the future, and those people might not know the reasons for your code suggestion.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.