Misc Network Tools is a set of miscellaneous tools I've had knocking around for over twelve years. With this release, though, I've added a few new ones that might be useful to some of you dealing with large sets of IP data, like blacklists or what have you.
ipconsolidate is a handy script that can take any kind of arbitrary text and extract all of the IPv4/IPv6 IPs and network addresses, eliminate redundant entries, and print a sorted list. What makes this a little special is that if, say, you have both 192.168.0.54 and 192.168.0.0/24 in the list, it will eliminate the single IP because it knows it's already part of the network range. It can also do the same for 10.0.200.0/24 and 10.0.0.0/8, as well.
ipgrep is a similarly capable grep. It can find lines in any input that logically match the address you're searching for. It will print lines that contain CIDR ranges that would encompass the IP you're looking for or, conversely, print all the lines with IPs that would be part of the CIDR range you're looking for. It also supports exact and quiet matching just like traditional grep.
This isn't just another Paranoid update this time. This time we have some significant enhancements of various APIs which continue to save me time on some of the more repetitive tasks.
Paranoid::Args, for example, has reduced the number of options required for option templates, with some being set automatically when obvious semantics are identified. On top of that, some of the more standard options now have importable templates.
Paranoid::Debug now provides constants for debug levels 1 through 8 which are intended for developer use and won't collide with Paranoid's internal debugging.
Paranoid::Filesystem has improved pmkdir's handling of permissions more predictable, and ptranslatePerms now allows you to pass translated and untranslated permissions with impunity, while still giving you the expected form out.
Paranoid::Log, however, has the most significant work done. It now supports both STDOUT and STDERR, which doesn't at first blush seem like much, but with a new plverbosity function, you can now map PL_DEBUG - PL_NOTICE log messages to STDOUT, and PL_WARN - PL_EMERG to STDERR with one call. This gives you a way to tune end-user verbosity for console messages without having to subject them to pdebug diagnostic input.
But, wait, there's more! All too often I've had to lace my code with dual calls to plog and pdebug. There's now a PDebug logging mechanism that can eliminate that redundancy, distributing both diagnostic messages to logs and STDERR as needed.
That brings us to one more significant enhancement. pdebug has long supported printf functionality with automatic array/scalar coercion and undef translation to strings. As long as you're enabling the PDebug mechanism, you can now leverage those same capabilities in plog, regardless of what logging mechanism those messages are distributed to.
Finally, like always, there are other small tweaks and improvements, but you'll need to peruse the changelog to get the full picture.
Another day, another Paranoid update. Mostly minor updates, one significant improvement (pfork()), and one embarrassing bug (pflock()). A better test would have shown how it wasn't working quite the way it should, so now we have that better test.
It is what it is, but, that said, I do have to say the rewrite is still paying dividends in convenience of use over the old rev. We might actually be getting somewhere with this. ;-)
Leif Sawyer pointed out a wee bug in the logic for NS server counts when the tool isn't run in strict mode. I also fleshed out more of the documentation. So, go get the latest zone-lint.
Life really gets in the way, sometimes. That said, I've found some time to brush some dust off an old script, and it may have some usefulness for you, as well.
zone-lint is a DNS zone lint check tool which provides some RFC-based validation of published DNS records. I've found it good practice to run this script after every update. I originally wrote this over ten years ago, but have now ported it to the latest Paranoid API, along with adding IPv6 support.
I couldn't very well leave Net::ICAP in a completely broken state while working on my applications, so I banged out a quick port, along fixing a few outstanding bugs reported by the good folks on rt.cpan.org. This module more than likely needs a complete review and rewrite, but one thing at a time.
Aaannnnd, just like that, here's another update. SREZIC@cpan.org caught my oversight in Parse::PlainConfig where I forgot to update the minimum required version in Makefile.PL for Class::EHierarchy. So, for that reason, P::PC gets a second release today.
All that said, I will have to admit some to small API alterations to Paranoid::BerkeleyDB, which is probaby bad form. I don't believe this module is widely used, however, so I'd rather make it now than wait. It does make the module behavior more consistent.
So, without further adieu, the following new modules have been released both here and on CPAN:
A quarter of a year is a long time to disappear. The last quarter of any fiscal year is always a killer as projects have to be wrapped up before the new budgets come in. Then, there's the holidays and a general desire to enjoy the real world with the family rather than stay mired in the digital sphere.
Regardless, I'm back. And I have been generally running amuck generating all kinds of extra work for myself. While working on autofwd, which I'm quite excited about, I happened to have snagged myself on several corner cases where the purported promises of Class::EHierarchy weren't quite delivered. The deeper I digged, the more it became apparent that I had really done a rubbish job of justifying that module's existence.
The only sane thing to do was to throw the entire code base out and start from scratch. I am rather ashamed of what ridiculous spaghetti code it was, I'm not quite sure how I let that malformed beast escape the lab. What was needed was a very methodical object build and destruction process, along with a much more thorough testing suite.
That said, I have good news and I have bad news. The good news is that Class::EHierarchy is complete. The bad news is that you can't have it, yet. Like the rewrite of Paranoid, the rewrite required an API break. Unfortunately for me, that also means that two of my more newer code updates now need to be updated to accomodate that.
End sum, because I don't want any psuedo-stable modules on CPAN broken I need to update them in order to do a coordinated release that keeps everything working.
Stay with me, the improved API(s) are worth it, and will benefit code quality built off of it. Thank you for your patience.
Small update to Paranoid for added functionality to Paranoid::Process. I added a signal dispatcher so I wouldn't have to worry about writing monolithic signal handlers. Now I can just add subroutines to a queue and have the dispatcher process them all as signals are recieved.
Also added minor updates to Microscopic Certificate Authorithy. No show-stoppers in terms of bugs, just some documentation updates and a correction to the logging mechanism.
Many tangents later it became apparent that others might get some use from a little package I've been using for years. Unfortunately, that meant refactoring the interface a wee bit and documenting it. Long story short, Microscopic Certificate Authorithy is now available.
Nothing too fancy, it's just an extremely simple way to get an internal CA up and running so you can quit adding a hundred exceptions in every browser you use. So simple that it only needs OpenSSL and GNU make in a UNIX environment to work.
And yes, I will get back to work on finishing my ports... :-P
The best laid plans of mice & men go oft' astray... Minor update this time for Parse::PlainConfig. Debian folks found a typo and I found a bug with the error reporting in the parser.
The Debian folks also find a trove of typos in Paranoid. Another gentleman (firstname.lastname@example.org) also pointed out the need for an improvement to the IPv6 regex, so that's there as well.
A few bug fixes and some expanded documentation necessitated the release of Paranoid v2.01. CPAN should be updated before long.
Not to sound like a broken record but I just uploaded Paranoid::BerkeleyDB to CPAN and this site. CPAN Testers pointed out a wee oversight of mine in which I was relying on a BDB 4.6+ call that failed on dists installed with older versions. Added some intelligence to fix that.
Just uploaded Paranoid::BerkeleyDB to CPAN and this site. Minor update, mostly documentation updates some small code adjustments.
Whew! Missed that by a mile, eh? So, I obviously didn't finish by the proposed deadline, but it is still coming. I've got a minor update to Paranoid::BerkeleyDB and I believe I will have everything in place to finish the port of autofwd.
This port isn't just a port, however, which is one reason why it's taking so long. It's essentially a complete rewrite meant to make it even easier to administer and far more flexible as well. With the help of Sander Klein as a sounding board I think this is going to be an update more than worth your while. It really can lend itself to so much more than just automated firewalling, but we'll see what people come up for it.
So, stay tuned, and thank you for your patience.
One of the joys of porting apps to the new API is in finding the little things we overlooked that make a big difference in usability. Found one such animal in Parse::PlainConfig in regards to a new feature of the parser: prototypes. As I'm reworking autofwd to leverage it it became obvious I was lacking an accompanying feature to make it really useful. So, now that's done. The update to the parser is live on this website now, and should be on CPAN within the day.
I'm not really trying to blow my deadlines, but I figured, hey, here's a program that's already using the new API. You may find it moderately useful: yapw.
Parse::PlainConfig is also live. This is updated to not only use the new Paranoid API but is a complete rethink on config parsers. This is tremendously nicer to use as a developer, but it does give up generation capabilities and the ability to parse arbitrary, free form configs. This now assumes that you have a well defined set of configuration parameters that are strongly typed by default. Much better suited for all of my use-case scenarios, at least. YMMV.
Paranoid v2.00 is now live, It should be up on CPAN soon as well. Please be aware that with this update I've also split out three modules into their own distributions since they required optional dependencies. This removes that ambiguity:
After quite the hiatus from posting it's time to get this site up to date. While I loathe updating web sites it's not like I haven't been coding, I've just haven't been sharing. If there ever was a time to share, however, it would be now.
I've spend most of the last ten years eating my own dog food, building most of my internal (and professional) code bases on my libraries. While those libraries have been effective, they haven't always been pleasant. As I've evolved as a hacker I've had to resign myself to dealing with a pragmatic reality that has refused, in many cases, to bend to my will.
Any of you who know me personally -- apologies for that, btw -- know that I've had a very long fixation on secure programming. All effective security has a cost and finding the sweet spot between effective security and usability is a constant balancing act. Both for the user and the developer.
I have been guilty for a long time of erroring on the side of the user. Not so much in usability -- again, sorry for that ;-) -- but in trying to guarantee effective security. It is in that obsessiveness that I managed to cross both the users and the developers. The best security in the world means nothing if it's so unpleasant that no one wants to use it.
All of which brings me to one of the most important lessons I could learn as a library developer: Don't be a dick, especially to other developers. It is perhaps ironic that as a cyber-recluse it took some time for me to recognize while slinging code that "that Paranoid guy's a dick." When I don't like using my own code, there's a problem. Therefore, I must reform.
I have spent considerable time over the last year rewriting my libraries, sometimes rewriting them entirely from scratch. The intent was to make the API more enjoyable to use while minimizing any adverse impact on the security goals.
For my Perl modules most of this is in the little details. Eliminating the use of function prototypes to make it easier to pass in arguments in list form, rather than explicitly by value. Or deploying prototypes only where it makes syntax more pleasant so the developer doesn't have to explicitly pass data structures by reference. By not croaking at every imagined offense, instead just exiting with a false return value.
With some of my foundational libraries already updated the race is now on to port the rest of the software that relies on it to the new API. That does mean, unfortunately, that software on this site is effectively broken if you grab the last revs of everything and try to build with it. My commitment to you is that I finish porting the broken software to the new API by the end of the month.
So there we have it. The ogre pokes his head out of his cave and has makes his proclamation. I have no intention of stopping my existence as an ogre, but in a small deviation I am very interested in feedback to these updates as I upload them. I am very excited about these changes, it's been a revelation -- to me, at least -- how much an API can affect one's eagerness to sling new code. The old APIs had really worn me down.
I'll leave it at that. Thank you for your time. Happy hacking!
Copyright © 1997 - 2016, Arthur Corliss, all rights reserved.