Clone Driving Behaviour

Clone driving behaviour using Deep Learning

With this behaviour cloning project, we give steering & throttle instruction to a vehicle in a simulator based on receiving a centre camera image and telemetry data. The steering angle data is a prediction for a neural network model trained against data saved from track runs I performed.
simulator screen sot

The training of the neural net model, is achieved with driving behaviour data captured, in training mode, within the simulator itself. Additional preprocessing occurs as part of batch generation of data for the neural net training.

Model Architecture

I decided to as closely as possible use the Nvidia’s End to End Learning for Self-Driving Cars model. I diverged by passing cropped camera images as RGB, and not YUV, with adjusting brightness and by using the steering angle as is. I experimented with using 1/r (inverse turning radius) as input but found the values were too small (I also did not know the steering ratio and wheel base of the vehicle in the simulator).

Additional experimentation occurred with using comma.ai, Steering angle prediction model but the number of parameters was higher then the nvidia model and it worked off of full sized camera images. As training time was significantly higher, and initial iterations created an interesting off road driving experience in the simulator, I discontinued these endeavours.

The model represented here is my implementation of the nvidia model mentioned previously. It is coded in python using keras (with tensor flow) in model.py and returned from the build_nvidia_model method. The complete project is on github here Udacity Behaviour Cloning Project

Input

The input is 66x200xC with C = 3 RGB color channels.

Architecture

Layer 0: Normalisation to range -1, 1 (1./127.5 -1)

Layer 1: Convolution with strides=(2,2), valid padding, kernel 5×5 and output shape 31x98x24, with elu activation and dropout

Layer 2: Convolution with strides=(2,2), valid padding, kernel 5×5 and output shape 14x47x36, with elu activation and dropout

Layer 3: Convolution with strides=(2,2), valid padding, kernel 5×5 and output shape 5x22x48, with elu activation and dropout

Layer 4: Convolution with strides=(1,1), valid padding, kernel 3×3 and output shape 3x20x64, with elu activation and dropout

Layer 5: Convolution with strides=(1,1), valid padding, kernel 3×3 and output shape 1x18x64, with elu activation and dropout

flatten 1152 output

Layer 6: Fully Connected with 100 outputs and dropout

Layer 7: Fully Connected with 50 outputs and dropout

Layer 8: Fully Connected with 10 outputs and dropout

dropout was set aggressively on each layer at .25 to avoid overtraining

Output

Layer Fully Connected with 1 output value for the steering angle.

Visualisation

Keras output plot (not the nicest visuals)

Data preprocessing and Augmentation

The simulator captures data into a csv log file which references left, centre and right captured images within a sub directory. Telemetry data for steering, throttle, brake and speed is also contained in the log. Only steering was used in this project.

My initial investigation and analysis was performed in a Jupyter Notebook here.

Before being fed into the model, the images are cropped to 66×200 starting at height 60 with width centered – A sample video of a run cropped.

Cropped left, centre and right camera image
Cropped left, centre and right camera image

As seen in the following histogram a significant proportion of the data is for driving straight and its lopsided to left turns (being a negative steering angle is left) when using data generated following my conservative driving laps.
Steering Angle Histogram

The log file was preprocessed to remove contiguous rows with a history of >5 records, with a 0.0 steering angle. This was the only preprocessing done outside of the batch generators used in training (random rows are augmented/jittered for each batch at model training time).

A left, centre or right camera was selected randomly for each row, with .25 angle ( for left and – for right) applied to the steering.

Jittering was applied per Vivek Yadav’s post to augment data. Images were randomly transformed in the x range by 100 pixels and in the y range by 10 pixels with 0.4 per xpixel adjusted against the steering angle. Brightness via a HSV (V channel) transform (.25 a random number in range 0 to 1) was also performed.
jittered image

During batch generation, to compensate for the left turning, 50% of images were flipped (including reversing steering angle) if the absolute steering angle was > .1.

Finally images are cropped per above before being batched.

Model Training

Data was captured from the simulator. I drove conservatively around the track three times paying particular attention to the sharp right turn. I found connecting a PS3 controller allowed finer control then using the keyboard. At least once I waited till the last moment before taking the turn. This seems to have stopped the car ending up in the lake. Its also helped to overcome a symptom of the bias in the training data towards left turns. To further offset this risk, I validated the training using a test set I’d captured from the second track, which is a lot more windy.

Training sample captured of left, centre and right cameras cropped

Center camera has the steering angle and 1/r values displayed.

Validation sample captured of left, centre and right cameras cropped

Center camera has the steering angle and 1/r values displayed.

The Adam Optimizer was used with a mean squared error loss. A number of hyper-parameters were passed on the command line. The command I used looks such for a batch size of 500, 10 epochs (dropped out early if loss wasn’t improving), dropout at .25 with a training size of 50000 randomly augmented features with adjusted labels and 2000 random features & labels used for validation

python model.py --batch_size=500 --training_log_path=./data --validation_log_path=./datat2 --epochs 10 \
--training_size 50000 --validation_size 2000 --dropout .25

Model Testing

To meet requirements, and hence pass the assignment, the vehicle has to drive around the first track staying on the road and not going up on the curb.

The model trained (which is saved), is used again in testing. The simulator feeds you the centre camera image, along with steering and throttle telemetry. In response you have to return the new steering angle and throttle values. I hard coded the throttle to .35. The image was cropped, the same as for training, then fed into the model for prediction giving the steering angle.


steering_angle = float(model.predict(transformed_image_array, batch_size=1))
throttle = 0.35

Successful run track 1

Successful run track 1

Successful run track 2

Successful run track 2

note: the trained model I used for the track 1 run, is different to the one used to run the simulator in track 2. I found that the data I originally used to train a model to run both tracks, would occasionally meander on track 1 quite wildly. Thus used training data to make it more conservative to meet requirements for the projects.

Advertisements

My first lane detection algo

I’m all for practical learning by building things. There’s nothing like getting stuck into a project and seeing results. Whilst a little progress is a good motivator, it also shows you how much it is that you don’t know.

I was pleased with the results of my first project doing the Udacity Self Driving Car Engineer Nanodegree. Yet what was more pleasing was being shown how much experimentation is really required. That is that is so much to learn.

This first module was about understanding some of the principles of computer vision that apply. We first started with Canny Edge Detection

and then Hough transform to detect lines within a region of interest.

The first project was to apply this learning, first to set of static images and then to a couple of videos captured whilst driving on a highway.

How cool is the final result .

I submitted my Jupyter Notebook for review. To pass you need to ensure that you meet specification. I passed here is the review feedback.

Some of my reflection thoughts on what can be improved in a future iteration of the project:

  • look for the road horizon starting from bottom centre of the image working up – asphalt has a fairly unique colour
  • break ROI into left and right lanes earlier – seems that at least with driving on a highway without lane changing that we can assume with confidence where they should start at the base of the image
  • segment each ROI into chained vertical blocks of a smaller width
  • when drawing connect intersection of cv2.fitLine lines
  • increase number of segments if lanes are curving left or right
  • label the lines with colour and type – continuous, dashed etc
  • feed the previous result into the evaluation of the next image
  • determine when an image has no lanes that could be considered reasonable
  • lane changing and entering a lane from a curb needs more though
  • if using smaller more specific left and right lane ROIs should allow for following a vehicle
  • not sure how rain affects this – might have to do a test and capture video in a tropical down pour this storm season
  • this approach wouldn’t work in snow. would require a different approach

Its still early yet in the nano degree but I’m hooked already. Happy coding and driving.

Congratulations – You’ve been Accepted

Congratulations – You’ve been Accepted

It was my birthday and a little email arrived from Udacity titled “Congratulations, you’re going to be one of our very first Self-Driving Car students!”. I had never thought with the number of people that were applying that I’d have a chance. Yet here was the email as a birthday present. I was both thrilled and apprehensive – I’d gotten in and now had to follow through.

I’ve been fascinated with robotics and AI for quite sometime now. And as always I seem to be too early with investigating business endeavours in Australia related to such. Whilst my cycling project was interesting, I was getting bogged down with having to learn how to use all the technology by myself. It always surprised me that I was able to get some results given how large the US teams seems to be that were using the same open source projects.

I’d applied to the program, because I thought I could leverage what I’d been learning through my previous endeavours around learnings from my cycling big data sensor project. The course had a focus on using things like ROS (Robotic Operating System), TensorFlow and Open Computer Vision. Plus it’d just seemed to be a great idea to work on technology related to self driving vehicles in a structured way. There is so much interest in this subject matter.

When I shared with my Facebook friends, that I’d been accepted, it was the most liked post I think I’ve ever had. Speaking people they say “wow thats great”. So its past the litmus test and I’ve just started.

The rise of chat bots and the fall of apps

It once was cool to build a mobile app and many startups in the past had built successful businesses with them and a minimalist web site. Now the chances of a new mobile app, creating mindshare and enabling a spot on a person’s home screen is next to impossible. We’ve reached peak app and new style of apps called chat bots are taking mindshare.

App stores are flooded, the majority of apps are rarely downloaded or found for that matter, as they do not rank.

It’s now harder than ever for a developer to build an app, that will replace the staple set of apps, a user does have on their devices. The frontier has changed to chat apps that have a conversational style interface either using text or voice (think siri). If you are building a new mobile app, stop! and reconsider how you are going to reach your target audience.

These new chat apps are leveraging existing instant messaging apps and agents on websites. Increasingly also APIs are being created and exposed to allow developers to interact with well known personal assistants like Siri. Some may argue that the interaction between human and computer is frustrating. I’d agree, having occasional back and forth sessions with Siri, to dial on my iPhone, a person I call regularly. However the situation is slowing improving as machine learning/AI technology improves behind the scenes.

Many will argue that we are not seeing anything new, that it is just the same technology and approaches that have been around for ages. The quest, as such to pass the Turing test where a judge can not determine if he or she, is talking to a machine or a person.

I think we’ve reached an inflection point, where a new class of conversation chat bot is being enabled by the gradual and constant exponential evolution of computing technology, sharing of open source component technology (such as natural language processing) in conjunction with the ongoing to quest to provide individually tailored answers to people’s own question through understanding the explosion of data available online.

This is also backed up by a dramatic increase in tech news coverage regarding startups in the US and with training/conferences covering this area.

So forget building a mobile app and start building a chat bot!

 

 

Switching off from Aussie innovation for the time being …

The slow dawn of reality has crept into my thinking, that what I’m presently witnessing, is the rise of politically correct innovation within Australia. That is, that there is a rush on, to be positioned, to secure funding and “innovation wash” existing service offerings, ready for when government programs come into affect.

My high hopes for an ideas boom have been dashed somewhat of late. Not so much from the intent, but from the reality, that the intent does not match reality. There is significant education (dare I say re-education) required.

Let me show you what I mean.

If we look at the innovation website Business.gov.au. (their definition here ), it basically suggests, that innovation is about change. It follows:

What is innovation?

Innovation generally refers to changing or creating more effective processes, products and ideas, and can increase the likelihood of a business succeeding. Businesses that innovate create more efficient work processes and have better productivity and performance.

Now if we look at the wikipedia innovation article  it suggests, The term “innovation” can be defined as something original and more effective and, as a consequence, new, that “breaks into” the market or society.

Innovation is a new idea, or more-effective device or process.[1] Innovation can be viewed as the application of better solutions that meet new requirements, unarticulated needs, or existing market needs.[2] This is accomplished through more-effective products, processes, services, technologies, or business models that are readily available to markets, governments and society.

So from my perspective the first one, is inwards looking, about the term innovation (as in thats an innovative idea) to do business change or continuous improvement.

I’ve often argued that changing a business process to make it more effective is not innovation. However if that idea, is bought to market, as a new product or services offering, then it is innovation ie there has to be diffusion into a market or society.

Now if we look at the slick new marketing or education material (your viewpoints may differ on this) being produced by National Innovation & Science Agenda Australian Government it refers to how Australians have been good at Ideas, but now we need to get better at commercialising –  turning those ideas, into new products or services.

As you can see, there is a long road ahead, with a lot of jargon presently such as “Ideas Boom”. It will take some time for people, to agree on what things mean (even though they have great definitions available now) and to reach consensus. Then decisions will need to be made about how much capital is to be made available and under what investment thesis it will be allocated.

There are a lot of people shouting about a number of things surrounding these topics and if your not shouting the politically correct message too then no matter how novel and disruptive your idea or invention is, it may not benefit from the “Ideas Boom”. If this is effecting you, jump on a plane and go to Silicon Valley or elsewhere that may be appropriate.

I keep hearing about the lack of opportunity here in Australia, in many fields on podcasts I listen to occasionally. On those podcasts, when people ask eminent panel members about their thoughts on the subject, invariably, their answer will be that we do hope you stay and help drive the next generation. It always surprises me, assuming that these persons are in tenured positioned, how devoid their responses seem to be, from the reality of needing money (or some may say capital) and support to do so. Its this later bit, that’ll take so long to grow here in Australia. It may also require a generational change. The notion of not taking risks in some is the antithesis of what is required in an “ideas boom” era.

So I’m thinking of slowly fading away from observing and commenting on all of this, until I need something concrete from it. Presently it all seems to be a nice discussion, but discussion is after all discussion and not tangible outcomes.

 

Nick’s tips for Silicon Valley

For Silicon Valley you need to find meet ups/events. That’s where people network.NASA cap smiling oct 2015

During the day a lot of persons are working on site in the large campuses of the tech companies. Traffic is fairly heavy on the main routes in the early morning with people still heading into work after 9 AM.

The wider Bay Area is really massive. Expect some decent travel times and I’d suggest on the first trip getting a hire car. I’ve used the CalTrain to get from San Francisco out to Mountain View. The people at Enterprise Mountain View are really friendly. If its your first time driving a left hand drive, the roads are also a little less busy than say San Francisco. So you can practice on side streets. Remember driver in the middle!

San Jose is a city that many, in Australia, would not of heard of. It has the highest per capita earnings of any city. Fuelled of course by tech. Theres not much to see there but its worth at least a visit.

Mountain View. I’d say is the heart of Silicon Valley. It has many of the well known tech companies head quarters.

Places to have a look at or around Mountain View:
– Red Rock Coffee Mountain View (peeps use it as a place to work)
– Hacker Dojo http://www.hackerdojo.com – they have tours on Friday nights etc you need to work out whats on via meetup
– Apple HQ, 1 Infinity Drive Cupertino
– Drive around Stanford Uni (really hard to get a park before 4 PM)
– NASA Ames Research Centre – small visitor centre with a shop
– Moffatt Field http://www.moffettfieldmuseum.org (get a few selfies in cockpits)
– Google HQ – driving near there might see a Google Self Driving Car. Theres a shop/visitor centre but think its only open Tue-Thu
– visit the Computer History Museum (well worth the time)
– Facebook HQ (nothing to see besides the campus and the FB like sign out the front)– Intel HQ has a good museum
– Yahoo, eBay etc all have office spaces but are pretty boring

A little bit off the beaten path  is finding Steve Jobs Garage and Hewlett Package Garages.IMG_8432

Nick’s tips for San Francisco

Ok .. be prepared for a mix in San Francisco. Your going to have homeless people all over the place with rich affluent tech people alongside. Its a big contrast. A big issue in the city is gentrification. Having said that, there are lots of really cool places but they might be a little hard to find. IMG_8149

Try foursquare app or trip advisor. The apps really do work here. They are your friend.

For the touristy type stuff, I head straight down to http://www.pier39.com/ and look at the sea lions. Then I jump on a one hour harbour tour. They all go by alcatrez and under the Golden Gate. This way I really know I’m in San Francisco. The other clue is the waft of dope floating down Market st. Apparently its legal to smoke for medical purposes in California, just your not supposed to do it in the open.

Its hard to get onto an alcatrez tour at this time of year (October as I write this), if you haven’t booked in advance. However some people like Alcatrez, others don’t. I’ve yet to get on the rock but it still fascinates me when I go by it, on a harbour tour.IMG_8160

Pier 39 is next door to Fishermans Wharf – restaurants etc. Try https://www.boudinbakery.com/ they are famous for their clam chowder in a bread bowl. I have breakfast at Boudin the first day in San Francisco now and go for the traditional egg and bacon brekkie.

Another good thing to do is to hire a bike and cycle over the Golden Gate bridge – its not hard. Most catch the ferry back from Sausalito (be weary of the time of day as big line ups to get on the ferry). If fogged in maybe choose another day or wait for it to clear. Plenty of bike hire places around $7 per hour.

If you want to hop on the Cable Car, I’d avoid doing it at power/market st turn around near IMG_8279union square. Lines are always long. Try doing it next door to Fishermans Wharf at http://www.ghirardellisq.com/. One of the first stops is the top of  Lombard St. Could walk down it and then catch the cable car again 🙂

San Francisco is next highest densest populated city after New York. I think the population itself is about ~750K. The broader bay area is massive – highway 101 is the main route to Silicon Valley. Theres heaps of meet ups and events on during the week. Its also a bit cheaper to stay then San Francisco. However not much to do around Silicon Valley (unless you want to work) – its really suburbia with massive campuses.

To find events use meetup,com or eventbrite. More then likely you’ll find an event that interests you like close by (SF and wider Bay Area). Distances outside of San Francisco are deceptive, so you really need a car or use say the caltrain/BART and uber.

ohh I  nearly forgot about the maker movement in SF http://www.techshop.ws/tssf.html – see if you can get a tour.

Theres also startup house nearby setup by australians. has bunk beds for $25 per night .. think i’m a bit over that lol but many people think its great.

The above is a slightly modified version of an email, I sent to a friend, explaining about what to do in San Francisco. If you have some other tips or if your a regular traveller to San Francisco, I’d love to hear about what rituals you now have.

Safe travels and happy exploring.