Forum New Posts Notifier for NiftyHost

[RichardGv]

The script hasn't been updated for almost half a year, since I'm too lazy to publish the revisions I made. And I think there's nobody else using my script except me right now. :) But here it is, anyway:
Nightly 20110408:
http://dl.dropbox.com/u/283669/python/ne...0110408.py
This revision includes many many changes so here's an annoyingly lengthy changelog:

1. Fixes the login issue of NiftyHost. Somehow the Cloudflare servers that you are using don't send cookies correctly when the script has a Python-ish user agent string. I added a fake Firefox user agent to deal with it. (Beware that this user-agent may break the operating system statistics of NiftyHost since it's a Linux x86_64 one. :) Don't consider that you have slept for 20 years (the future belongs to Linux!) when you wake up and find 60% of visits to NiftyHost forums come from Linux machines, Zack.)
   
2. Added custom HTTP header support.
   
3. Added DNS cache support to improve the speed for a little bit. Very ugly hack it is, though.
   
4. Better exception handling. The script is less likely to (yet still could possibly) exit automatically when it encounters an unhanded exception.
   
5. Added login formhash support for Discuz forums. Support for the "UCWeb forum" was added since one of my friends requested so, but it cannot check for new posts since seemingly Discuz doesn't support read/unread system at all! So essentially it can be used to periodically DDOS a Discuz forum only. :)
   
6. It should now be cross-platform out-of-box. I added platform detection to the script. This brings the possibility to, for instance, run the script from a Dropbox directory that is shared between a Windows system and a *nix one.
   
7. On *nix systems it prints messages with colors (ANSI escape sequences) by default.
   
8. Improved the modularity of the script so now it's easier to customize it.
   
9. More comments about the configuration options. I found writing comments harder than writing code since I always have to consider if an expression is appropriate in English -- which is my second language. :)
   

And nah, this is still nightly. It may cook a fried CPU as your breakfast "automagically" and "for free". Use it at your own risk! :D

[HiddenKnowledge]

Don't consider that you have slept for 20 years (the future belongs to Linux!) when you wake up and find 60% of visits to NiftyHost forums come from Linux machines, Zack.

I doubt Zack will look at the logs.. He doesn't have access to them yet :P

[szdbuy]

Yeah, I can use py2exe or cx_freeze to pack it, but:

[HiddenKnowledge]

Yeah, I can use py2exe or cx_freeze to pack it, but:

But, what? Lol..

It may cook a fried CPU as your breakfast "automagically" and "for free".

Wow, free breakfast? O_O''

[RichardGv]

Nightly, April 28th, the version that have taken me days of effort:
https://github.com/richardgv/newpostsche...tscheck.py

• Sad news firstly: This version requires Python 3.2, since it utilized the argparse library introduced in the version. Users have to upgrade the Python interpreter. This is non-negotiable.
  
• External configuration files are now supported, i.e. from now on you need not to modify the script itself to add your usernames and passwords or modify any settings (actually, most settings). Configurations are stored in XML format (in order to make it "eXtensible"), parsed with xml.parsers.expat, and can be generated by the script using ElementTree or lxml library. lxml is a widely-used faster and more powerful XML processing library, which is not included in Python package itself, and Windows users may install it with the installers that you can find in the link, while the Linux/BSD users... Well, you know it. When the script fails to load lxml, it would use the ElementTree library as fallback, which lacks the support of CDATA sections.
  

  How to generate a skeleton configuration file?
  For *nix users it's easy:
  

  Code:

  python3.2 newpostscheck.py -o niftyhost -g newpostscheck.xml

  For Windows users it would be much a hassle to finish the same task. You generally should launch a Command Prompt (or PowerShell, or whatever these guys decide to name it in the future), and enter the following commands:
  

  Code:

  cd PATH_TO_THE_SCRIPT_FILE
c:\python32\python.exe newpostscheck.py -o niftyhost -g newpostscheck.xml

  (Refer to the section below for what these bizarre switches stands for.)
  Then please open the newpostscheck.xml in the directory in which the script file is placed, with a text editor fully supporting UTF-8. In the file you would find lines like this:
  

  Code:

  <target name="username" key="niftyhost"><![CDATA['']]></target>
  <target name="enable" key="niftyhost"><![CDATA[True]]></target>
  <target name="password" key="niftyhost"><![CDATA['']]></target>

  (If you don't have lxml installed, the format of the XML elements would be different, and there won't be CDATA section inserted, but the basic structure is the same. If you don't use CDATA to wrap the text content, you need to escape "&" and a few other characters. Pay attention to the "key" attribute and make sure it's indeed "niftyhost" that you are editing when you read the XML file.)
  Basically in the "username" and "password" field, you should fill in your username and password enclosed in single quotes, like:
  

  Code:

  <target name="username" key="zemhost"><![CDATA['RichardGv']]></target>
  <target name="enable" key="zemhost"><![CDATA[True]]></target>
  <target name="password" key="zemhost"><![CDATA['MY_PASSWORD']]></target>

  Save the file, launch the Python script, and it's done!

  With the option -G you may generate a full configuration file and view all the settings you may change. You may modify the regular expressions with which the page matching is performed, configure custom commands to execute after a new post is discovered, localize the messages the script prints, or even add new forums to the list of forums the script currently supports (which requires a lot of work. though). I may write a better document describing all the available settings in the future.
  
• Commandline arguments support. (argparse is awesome and foolproof, by the way.) Use the "--help" command switch to view an introduction about all supported commandline arguments.
  
  Code:

  usage: newpostscheck.py [-h] [-d] [-D] [-e TARGET [TARGET ...]]
                        [-E TARGET [TARGET ...]] [-o TARGET]
                        [-g FILE | -G FILE | -s]
                        [CONFIGURATION_FILE [CONFIGURATION_FILE ...]]

Checks for new posts in various forums

positional arguments:
  CONFIGURATION_FILE    path to the configuration file ("-" for stdin)

optional arguments:
  -h, --help            show this help message and exit
  -d, --debug           enable debug mode
  -D, --no-debug        disable debug mode
  -e TARGET [TARGET ...], --enable TARGET [TARGET ...]
                        enable a target
  -E TARGET [TARGET ...], --disable TARGET [TARGET ...]
                        disable a target
  -o TARGET, --only TARGET
                        keep only a target enabled
  -g FILE, --genconf FILE
                        generate a basic configuration file ("-" for stdout)
                        and quit
  -G FILE, --genfullconf FILE
                        generate a configuration file containing all the
                        available settings ("-" for stdout) and quit
  -s, --list-targets    list supported target sites
  The environment variable NPC_OPTIONS (Well, it's called NPC since it's New-Posts-Check script, which has nothing to do with the alternative NPCs.) can be used to add default options to the script, e.g. NPC_OPTIONS='-d' turns debug mode on by the default. It's useful on very specific conditions and is placed there as a courtesy for the crazy programmers among you. Specfying path to the configuration directory in it or use syntax like "--" may cause unpredictable results (for me, not for the computer) that I cannot be responsible for. Do not play with fire in it, or you will burn your fingers. :D
  
• Commands executed after new posts are discovered are now queued by default. So if you have specified a command that takes quite a long period of time to execute (e.g. mplayer), the script won't be throttled. Using the subprocess module may create more ideal results, yet it's beyond the scope of a simple Python script.
  
• Due to HK's request, URL-specific HTTP header supported is introduced. Review config['headers_hosts'] for more information.
  
• Gzip compression is now supported to save some bandwidth (and money!) for you. It just sits in the background working silently and you need not to worry.
  
• Many other changes and enhancements that I'm tired to list. Writing README is more painful that writing the code itself...
  

Again, this script could be buggy since it's not tested enough.

I've uploaded the script to GitHub, and you can keep track of the latest development snapshots there. Well, actually what I am releasing all above are development snapshots, aren't they?
https://github.com/richardgv/newpostscheck

I will update the main thread tomorrow. Sleepy today. :)

[RichardGv]

Nightly, April 30th, mostly a bugfix release:
https://github.com/richardgv/newpostsche...tscheck.py

• Fixed a few grammar/vocabulary mistakes in the --help message.
  
• -g parameter now generates configuration entries for 'msg_newpost', 'cmd_newpost' and 'cmd_err', so you can now configure the message displayed after a new post is detected, and the commands displayed after a new post is detected or an error is encountered more easily. (In the last version you have to use -G to generate entries for these settings. Cases do count, and -g isn't -G, by the way.)
  
• More graceful file object closing in configparse() and debug_file()
  
• Configuration file directory recursion is now performed in explicit alphabetical (actually ASCII) order, so the recursion now works in a fontconfig fashion, and you can create a directory that contains configuration files like "10-no-debug.xml", "20-mplayer-err.xml", etc. and specify the directory as a "configuration file" on the command line and it will work.
  

[zyzyyshy]

That's kinda neat!

[RichardGv]

Nightly April 4th:
https://github.com/richardgv/newpostsche...tscheck.py

Since April 30th some changes has been made. Including bug fixes and new command arguments. Read the commit log if you wish to know more. I'm too lazy to write a changelog. :)

The most important addition (for most of you :D ) is there's a README file now. It offers brief introductions for some previously undocumented features, and actually there are more undocumented features laying in the script... (Seemingly GitHub doesn't render Asciidoc perfectly. If possible you may convert the Asciidoc source to HTML locally, provided that you do know what Asciidoc is. :D )

[ibin]

good i think so

[elsonkwok]

it's so long I will spend all day to learn that. thx