Tuesday 26 July 2011

[BOT] RSS and XFML news feeds

RSS News feeds

The RDF Site Summary format is an XML listing of the latest 'interesting' articles found. It is updated once every few minutes.

  • RSS feed most recent 15 items
  • RSS feed most recent 25 items
  • RSS feed most recent 100 items
  • RSS spec

    http://drugpolicycentral.com/bot/news/latest.xml (cached RSS 1.0).

    The system scans many online papers and news sources daily. Online news sites are crawled continually. You may poll as often as needed.

    Atom feeds

  • Atom feed most recent 15 items
  • Atom feed most recent 25 items
  • Atom feed most recent 100 items

  • Atom is another XML-based news feed format.

    XFML Feeds

    Also available are XFML feeds. XFML (eXchangeable Faceted Metadata Language) is a standard markup that can express some of the rich categorization and indexing that bot automatically performs on breaking news.

  • XFML feed most recent 25 items
  • XFML feed most recent 100 items
  • XFML feed most recent 200 items
  • XFML spec

    URL Parameters, Switches

    Options which change the output.
    url setting effect applies to
    HTML RSS Atom XFML JavaScript
    xml=1 output as RSS (1.0) RDF XML   X      
    rss=0.9
    or, rss=0.91
    or, rss=1
    or, rss=2.0
    output specific version of RSS XML   X      
    xml=atom output as Atom (XML)     X    
    xfml=1 output as XFML lightweight topic (map) XML       X  
    javascript=1 output as javascript         X
    max=N output at most N items X X X X X
    q=querystring search for querystring X X X X X
    concept=conceptname search for conceptname X X X X X
    srclink=1 link to original source site   X X X X
    noconcepts=1 in the descriptions, omit the listing of concepts   X X X X
    merge=1 try to eliminate apparent duplicates by selecting the most recent article X [1] X [0] X [0] X [0] X [0]

    Examples:

    http://drugpolicycentral.com/bot/index.cgi?xml=1&merge=1&concept=cannabis
    Latest cannabis articles, as RSS (default RSS 1.0). Merge to eliminate duplicates (i.e., the same article from different sources).


    http://drugpolicycentral.com/bot/index.cgi?rss=2.0&merge=1&concept=psychedelic
    Latest psychedelic articles, as RSS 2.0, eliminate duplicates.


    http://drugpolicycentral.com/bot/atom
    (Default) Atom feed of latest articles.


    http://drugpolicycentral.com/bot/index.cgi?xml=atom&concept=methamphetamine
    Latest methamphetamine articles, as an Atom feed.

    Javascript

    Another way to use our news feeds is to use Javascript. The javascript code snippet below will load up headlines, and print them out separated by linebreaks. Just copy and paste the following into html pages wherever you want to see the headlines. (See the table above for options that apply to javascript news feeds.)
    ' + title[count] + '<BR>'); document.write(description[count] + '<BR>'); document.write('(' + date[count] + ' - '); document.write(domain[count] + ')'); document.write('</P>'); } </script>
    Available Javascript information for each item:

    array namecontents
    titleitem's title
    linkurl to item
    descriptionshort summary/description of item
    datedate system found the item
    conceptslist of concepts detected in the item
    domainsource domain of item

    How the feeds are created

    The articles' text is downloaded, categorized, transformed, and summarized. Article text transformation consists of re-formatting, concept detection, classification, and mark-up. News feeds are served on demand. Newsfeeds for searches and for specific concepts are easily created. (Note the RSS link on searches.)

    The concept dictionary defines the "concepts" used for the classification and markup process.

    More RSS, XFML

    Additionally, all site searches are available as RSS, Atom, or XFML feeds.. Simply perform a search; the corresponding XML, XFML, etc. for that search is indicated by a link at the top of the search result page [this search as RSS feed XFML feed].

  • Mapinc.org XML/RSS News feeds
    About 150 drug-policy related RSS feeds. Grouped by Drug-policy topic, and also grouped by area. All items are selected by drug-policy analysts (mapinc editors) for relevance to drug policy.

    see also:

  • RDF Rich Site Summary (RSS)
    RSS background, history, links

  • xfml.org
    eXchangeable Faceted Metadata Language.
    more>>>
  • other notes
    My notes