PlanningAlerts.org.au

Willoughby Council scraper

Details

  • Type: New Feature New Feature
  • Status: Open Open
  • Priority: Minor Minor
  • Resolution: Unresolved
  • Affects Version/s: None
  • Fix Version/s: None
  • Component/s: Scraper
  • Labels:
    None

Description

I've got a nearly finished scraper for Willoughby based on the Wollongong one. Needs some cleanup. Just noting it here in the meantime. Can assign to me if you like.

  • Roger

Activity

Ascending order - Click to sort in descending order
Hide
Permalink
Henare Degan added a comment -

Hey Roger - I've just assigned this to you as requested.

I've also noted that you now have "developer" rights on this ticket tracker so that you can now make changes like this too.

Cheers, Henare

Show
Henare Degan added a comment - Hey Roger - I've just assigned this to you as requested. I've also noted that you now have "developer" rights on this ticket tracker so that you can now make changes like this too. Cheers, Henare
Hide
Permalink
Roger Barnes added a comment -

Attaching patch...

  • Add Willoughby to scraper_factory
  • Move common epathway code from wollongong_scraper to new epathway_scraper (abstract class)
  • Introduce new willoughby_scraper
Show
Roger Barnes added a comment - Attaching patch...
  • Add Willoughby to scraper_factory
  • Move common epathway code from wollongong_scraper to new epathway_scraper (abstract class)
  • Introduce new willoughby_scraper
Hide
Permalink
Matthew Landauer added a comment -

Awesome. Thank you. Will look at the patch as soon as I can.

Show
Matthew Landauer added a comment - Awesome. Thank you. Will look at the patch as soon as I can.
Hide
Permalink
Matthew Landauer added a comment -

Hi Roger,

Your new scraper for Willoughby works a charm! Thank you.

I did run into an error when I ran the updated scraper for Wollongong:

$ ./scraper_output.rb wollongong 2010-08-20
./scrapers/wollongong_scraper.rb:42:in `applications': undefined method `search' for nil:NilClass (NoMethodError)
from ./scrapers/wollongong_scraper.rb:30:in `map'
from ./scrapers/wollongong_scraper.rb:30:in `applications'
from ./lib/scraper.rb:57:in `results'
from ./lib/scraper.rb:52:in `results_as_xml'
from ./scraper_output.rb:38

Could you take a look?

Thanks!

Show
Matthew Landauer added a comment - Hi Roger, Your new scraper for Willoughby works a charm! Thank you. I did run into an error when I ran the updated scraper for Wollongong: $ ./scraper_output.rb wollongong 2010-08-20 ./scrapers/wollongong_scraper.rb:42:in `applications': undefined method `search' for nil:NilClass (NoMethodError) from ./scrapers/wollongong_scraper.rb:30:in `map' from ./scrapers/wollongong_scraper.rb:30:in `applications' from ./lib/scraper.rb:57:in `results' from ./lib/scraper.rb:52:in `results_as_xml' from ./scraper_output.rb:38 Could you take a look? Thanks!

People

Vote (0)
Watch (0)

Dates

  • Created:
    Updated:

Time Tracking

Estimated:
2h
Original Estimate - 2 hours
Remaining:
0m
Remaining Estimate - 0 minutes
Logged:
2h
Time Spent - 2 hours
Atlassian JIRA (v4.4.3#663-r165197) | Report a problem
Powered by a free Atlassian JIRA open source license for OpenAustralia Foundation. Try JIRA - bug tracking software for your team.