[cryptome] Re: NYC Taxicab Log Dump

  • From: doug <douglasrankine2001@xxxxxxxxxxx>
  • To: cryptome@xxxxxxxxxxxxx
  • Date: Thu, 26 Jun 2014 21:30:12 +0100

Excellent...I am very pleased to have added to your depth and breadth of education...as you have added to mine. And I am very pleased that we hold at least one author in common. I am always grateful when someone adds to my education or makes me think or act in a new way.


Can I also suggest a book by Norman Mailer...a bit dated now, and of course he died the other year without writing the sequel. It is called "Harlot's Ghost", available at all good bookshops...and a used edition is very cheap at Amazon...if I am allowed to advertise such things here on the list...though one can also get it on hire from ones local library for free...if there is such an organisation these days.

In the book you will discover a number of things about the CIA and such like. How the organisation is run, the compartmentalisation and how it works for and against the efficiency of the organisation and how it protects those who don't want to be brought out into the heat, the names of the families who were involved in its creation, their ancestors and descendants who are still involved (pseudonyms of course) and such like. Mailer spent many years researching it, and it is a bit dated now, but the essence is still there and it is a fascinating story. Puzzle Palace was my first introduction to the intelligence game and after reading it, I was hooked.

Also, there is something in there which helps all good spies...or dare I say, "spy buffs" in the book to memorise things like phone numbers and various other kinds of numbers, data and personal information and such stuff. None of this stuff is new of course, neither is it classified, and one can find a plethora of websites on line which offer various methods of improving one's memory, though one should never make the mistake that such improvement attaches to ones intellect and also, like many aspects of cryptography and security, there is a lot of snake oil around, which costs a lot of money. I am one of those who believes that mind mapping and the use of memory techniques can save one a lot of time, as well as helping with assimilation and retention of information; but I have never been able to make up my mind whether they in fact do so, or are just a trick to get one to spend some money. I am a great believer that knowledge should be free and open to everyone who cares to educate themselves.

Mind you the Ancient Greeks and Romans used such methods, and there are a number of biographies which suggest that great leaders used such skills to remember things like battle plans or complex algorithms or even the names of every soldier in their armies, such as Alexander...however, he died early, so such knowledge didn't save him.

On 26/06/14 21:06, Aftermath wrote:
OK :) and no, Ill have to look into Le Carre's novels.

And I prefer William ;)




On Thu, Jun 26, 2014 at 12:31 PM, doug <douglasrankine2001@xxxxxxxxxxx <mailto:douglasrankine2001@xxxxxxxxxxx>> wrote:

    Dear Aftermath...or is it Bill,

    One of the things that I learned early in life, particularly when
    I moved to London and then to the Isle of Wight, was that taxi
    drivers are a mine of information on the local environment.  When
    one is contemplating moving into a new area, then what can be
    better than entering into conversation with one's taxi driver to
find out who is who and who does what in the local environment. Of course, one doesn't have to enter into conversation with one's
    cabbie, after all, one is a captive audience, one just has to
    guide the conversation with a few "open" questions which helps to
    ascertain the kind of environment and type of people one is going
    or hoping to meet up with if one settles there.  The nice thing
    about such open source messages and conversations is that they, up
    until now, is that they were private and not recorded; and can
    establish so much of what one wants to know, without any
    additional expense rather than the hire. Of course, nowadays, the
    metadata is just as important...(if not more so)...than the
    content of the conversation, but that is the internet and all this
    new information technology for you.

    When I visited New York for a holiday...before 9/11, I met up with
    a cabbie whose English was very poor.  He was a Russian immigrant,
    yet knew his way around New York so well, and was able to tell me
    and me missus the best places to go and which places to avoid and
    at what time.  All it cost was a nice tip at the end of the
    journey, with which he was very pleased, and he could see that us
    Scots were not at all prejudiced about his origins or his
    politics.  What more can one ask for in life...sarcasm is such an
    ugly word...to be accused of using it is so hurtful...;-) .
    ATB
    Dougie.
    P.S. Have you ever read any of Le Carre's novels?  He has written
    23 and like all good security and intelligence buffs, I have read,
    nay, studied all of them.  You can find them in all good bookshops
    and they are available on line very cheap.  Like le Carre, I find
    fiction far more educational than non-fiction.  And fiction has
    the added advantage that it is not classified or subject to
    secrecy laws, yet contains all the craft of the trade of
    information.  Beware of course, that le Carre's main theme, as is
    that of William Boyd, is betrayal...


    On 26/06/14 19:54, Aftermath wrote:
    ... cant tell if sarcasm or not..?


    but yeah, I just found the link and I thought this list would
    appreciate the share :)


    On Thu, Jun 26, 2014 at 11:48 AM, doug
    <douglasrankine2001@xxxxxxxxxxx
    <mailto:douglasrankine2001@xxxxxxxxxxx>> wrote:

        On 26/06/14 18:19, Aftermath wrote:
        Dear Aftermath...or is it Bill?

        I thought this posting was absolutely brilliant. My
        congratulations to you for bringing it to our attention and
        Mr. Wong wh did the initial research.  Why?  Because...if you
        think on it, you have saved the NSA, the CIA and the
        intelligence and security departments  of the New York Police
        Force huge amounts of money on expenditure of resources...and
        foreign organisations such as the Chinese, the Israelis, the
        Russians and any ole Tom, Dick and Harry of a security
        organisation a lot of time.  Even the United Nations security
        and intelligence services will be all agog at your posting
        when they read it.  I hope you are not claiming copyright on
        it...;-) . At the click of a button, they can all find out
         who has been doing what in New York....United Nations
        officials, Ambassadors, envoys, consuls and all sorts of
        public and civil service officials, secret service,
        intelligence services, even the private sector.  All this
        metadata available to link up with known associations and
        links...and all open source too, who visited who and when,
        who was where and when, with a little help from the smart
        mobile phone, the most valuable source of i.d. location and
        contacts, you have provided the world, with a wealth of
        information.  Just think...the amount of money you have saved
        the world's intelligence organisations.  Absolutely f*cking
        brilliant....My congratulations...
        ATB;-)
        Dougie.
        P.S. Who needs privacy and encryption when we have all this
        stuff about the world leaders in our very own hands.  It is
        all a question of pressing the right button...


            http://chriswhong.com/open-data/foil_nyc_taxi/
            and
            https://medium.com/@vijayp/f6bc289679a1


            from the second link...

            "
            Recently, thanks to a Freedom of Information request,
            Chris Whongreceived and made public a complete dump of
            historical trip and fare logs from NYC taxis. It’s pretty
            incredible: there are over 20GB of uncompressed data
            comprising more than 173 million individual trips. Each
            trip record includes the pickup and dropoff location and
            time, anonymized hack licence number and medallion number
            (i.e. the taxi’s unique id number, 3F38, in my photo
            above), and other metadata.

            These data are a veritable trove for people who love
            cities, transit, and data visualization. But there’s a
            big problem: the personally identifiable information (the
            driver’s licence number and taxi number) hasn’t been
            anonymized properly — what’s worse, it’s trivial to undo,
            and with other publicly available data, one can even
            figure out which person drove each trip. In the rest of
            this post, I’ll describe the structure of the data, what
            the person/people who released the data did wrong, how
            easy it is to deanonymize, and the lessons other agencies
            should learn from this. (And yes, I’ll also explain how
            rainbows fit in).


            The NYC taxi data consist of a number of CSV-files.....
            "







Other related posts: