Posts by Hugues

Hi Hussein,

I did tried the importation into a collection and it worked well at first, but as the collection needs to be updated every day I had to deplore some sql crashes issues after several updates. It would have been the easyest solution.

If i can consider importing all of my external contents into a single searchable bunch, your solution will definitely be better.

Thanks.

Hi,

For some reasons my Perch Runway site retrieves a part of its informations from an XML source and also from the JSON stream of an IOS/Android app.

The XML is acquired every day using a cron job, I considered for a moment to import the XML resource into a collection to make its content searchable, but it's not a good idea considering the number of entries daily refreshed.

The JSON resource is lively acquired at the app.

I tried to implement a crawler to build the data into a mysql table for search purpose, but it's not really effective.

My question is: what do you think would be the best approach to make these external resources searchable by the Perch search application?

Thanks in advance.

Hugues

I just succeed selective importing just by using this formula:

$items = $Importer->find_items([

'filter' => 'id',

'value' => $offreid,

]);

if (empty($items)) {

$Importer->add_item([

'id' => $offreid,

'titre' => $titre,

]);

}

By now I have to add the udating and deleting capabilities and I think it will be fine.

Thanks.

If the numbering increases daily by several thousand units, that's indeed what I'm afraid of.

Hi,

This is not really good practice in my opinion, but the following process has been enacted by a third party and I am in the testing phase for its use.

For the purposes of a communal events calendar managed by this third party organization, it is agreed that each day using a cron job, an XML file is compiled and extracted from their database.

In order to use this resource within Perch, I implemented the following system:

Using a second cron job, I automatically import the contents of the XML file into a Perch collection.

Every day in this XML file, some items remain in place, but new items may appear and others may disappear.

Thanks to Perch's import system, I know that I can selectively update and delete the contents of the collection compared to each daily import, but I have a problem adding new items.

For now, the most effective way I've found is to drastically empty the collection and re-import everything every day. It works pretty well, but my concern is that the collection items id are not reset with the deletion and the identifier numbering is constantly growing.

The XML file is quite heavy (about 25Mo of dayly data on average) and contains hundreds of elements, which gives rise to id numbers that quickly make you dizzy.

My question is there a way for the collection items id in my process to be reset after deletion?

Any ideas are welcomed.