PlanningAlerts.org.au

Some scrapers appear to be having problems

Details

  • Type: Bug Bug
  • Status: Closed Closed
  • Priority: Critical Critical
  • Resolution: Incomplete
  • Affects Version/s: None
  • Fix Version/s: None
  • Component/s: Scraper
  • Labels:
    None

Description

I noticed that Logan didn't return any DAs in yesterday's run:

"...
[155] => Scraping authority Kogarah from http://www.planningalerts.org.au/scrapers/kogarah?year=2010&month=3&day=21
[156] => Found 0 applications for Kogarah
[157] => Scraping authority Lake Macquarie from http://www.planningalerts.org.au/scrapers/lake_macquarie?year=2010&month=3&day=21
[158] => Found 0 applications for Lake Macquarie
[159] => Scraping authority Logan from http://www.planningalerts.org.au/scrapers/logan?year=2010&month=3&day=21
[160] => Found 0 applications for Logan
[161] => Scraping authority Manningham from http://www.planningalerts.org.au/scrapers/manningham?year=2010&month=3&day=21
[162] => Found 0 applications for Manningham
..."

But if you run it manually: http://www.planningalerts.org.au/scrapers/logan?year=2010&month=3&day=22 you get a whole bunch of applications.

Because these scrapers can take a date argument, and we only ask for a particular day each day, I suspect a whole bunch are missing and alerts haven't been sent.

Activity

Hide
Henare Degan added a comment -

Ticket too vague - who submitted this?!

Show
Henare Degan added a comment - Ticket too vague - who submitted this?!

People

Vote (0)
Watch (0)

Dates

  • Created:
    Updated:
    Resolved: