New-school Security Ruminations

An older presentation on a 'new' topic

2008-06-24T14:56:00.000-07:00

Back in 2005 I gave this presentation at Black Hat Japan:

http://www.blackhat.com/presentations/bh-jp-05/bh-jp-05-brezinski.pdf

This year a number of security researchers have done work on the topic and are making the rounds with their presentations.

Just a few days ago there was a big to do about security vulnerabilities in ruby. The primary vulnerabilities were integer overflows affecting memory allocations in Array and String. However, it was pointed out to ruby-core in early 2006 that integer overflow issues existed in Array and that the memory allocation macros used through out the interpreter code were subject to integer overflows:

http://blade.nagaokaut.ac.jp/cgi-bin/scat.rb/ruby/ruby-core/7818

Great to see the communities getting ahead of the curve :/

Dynamic attribute validation for forms

2007-05-04T16:39:00.000-07:00

Developing usable web application is a lot more difficult than many people think. Forms are an obvious point of friction for users, so anything you can do to help get the user through the form successfully is a good thing. One of my applications has a fair amount of restrictions on data that the models will accept (though validation restrictions). Users were frustrated after submitting a form and having it rejected due to failed validations. This presented a couple of challenges.

-There are a couple of AR models that are updated or created from the form

-The validation and feedback to the user should occur as each field is entered, and any error messages should be presented close to the offending field

-The validations on the models are somewhat complex

Here is a flexible, but somewhat expensive, solution I came up with and plopped in a controller. Substitute the AR models in your application for Model1, Model2, Model3 in the following code:


private

  def validate_input_by_model
    error_msgs = Hash::new
    [Model1, Model2, Model3].each do |klass|
      classname = klass.to_s.downcase
      error_msgs[classname] = Hash::new
      if params.has_key?(classname)
        obj = klass::new(params[classname])
        unless obj.valid?
          obj.attributes.each do |k,v|
            if params[classname].has_key?(k)
              error_msgs[classname][k] = obj.errors.on(k)
            end
          end
        end
      end
    end
    error_msgs
  end

public

  def validate_field
    if request.xml_http_request?
      error_msgs = validate_input_by_model
      render :update do |page|
        error_msgs.each_key do |classname|
          error_msgs[classname].each do |k,v|
            page.inject_error_msg_if_condition "#{classname}_#{k}", v, v
          end
        end
      end
    else
      render :nothing => true, :status => 401
    end
  end

An example of the view code:


<% css_form_for :model1, @model1obj, :url => { :controller => 'controller1', :action => 'confirm' } do |@f| %>
  <%= @f.text_field :attribute1, :extra => 'Required' %>
  <%= validate_remote_by_model 'model1_attribute1' %>
  <% css_fields_for :model2, @model2obj do |@model2_f| %>
    <%= @model2_f.text_field :attribute1 %>
    <%= validate_remote_by_model 'model2_attribute1' %>
  <% end %>
<% end %>

The Helper (for the controller associated views) code:


  def validate_remote_by_model(id)
    validate_local_by_regex(id, /^[^&]*$/, '&amp; characters are not allowed. ') + observe_field(id, :with => ("%s[%s]" % id.split(/_/)), :url => { :action => 'validate_field' })
  end

  def validate_local_by_regex(id, regex = nil, error_msg = nil)
    default_regex, default_error_msg = *@@validations[@@validation_mapping[id]]
    regex ||= default_regex
    error_msg ||= default_error_msg
    function = <<-SCRIPT
    if (value.match(#{regex.inspect}))
      { Element.remove('l_#{id}_error'); } 
    else 
      { var prev_error = document.getElementById('l_#{id}_error');
        if (prev_error != null) {
          Element.update('l_#{id}_error', '#{error_msg}');
        } else {
          new Insertion.Before('#{id}', '<div class=\"field_error\" id=\"l_#{id}_error\">#{error_msg}</div>');
        }
      }
    SCRIPT
    observe_field(id, :function => function)
  end

This code is dense, but the object is to make the view code as simple and clear as possible and still be able to use the pleasant form_for (or css_form_for with plugin) helpers.

The helper code is knarly, since it is generating javascript, but its functionality is pretty simple. The field text is first check in the browser using a regular expression to screen for ampersands (see post about RoR/prototype issue), and then an AJAX call is made to validate_field sending the field and its value formatted in the standard ROR form syntax. The RoR form parameter syntax is model[attribute]=value. The controller code takes over from there and does its magic to see if the field value will pass validation when applied to the corresponding model. If validation fails and error messages are generated, the errors are passed back to the AJAX code. The javascript then inserts a new element in the DOM that is adjacent to the form field and contains the error message. The new element is updated or cleared based on the response to subsequent calls.

You can disregard that *@@validations[@@validation_mapping[id]] reference in validate_local_by_regex. It is simply a dictionary of regular expressions and error messages used for other in-browser validations. It is not used in this case.

Safe model attribute updates using AJAX

2007-05-04T13:40:00.000-07:00

Sometimes it is nice to have little AJAX click to edit controls that update specific attributes in a model. There are many ways to implement this functionality, some more DRY than others. The code below references an example application that contains users with a corresponding User model. Once authenticated, the user object is stashed in the session. In the following code, current_user references the instance of User for the currently authenticated user. One might be tempted to do the following in the corresponding controller:


  def method_missing(name)
    match = /\Aupdate_([a-z]{1,20})\Z/.match(name.id2name)
    if match && current_user && current_user.attributes.keys.include?(match[1])
        if params[:value]
            current_user.update_attribute(match[1],params[:value])
            current_user.reload
            render :text => current_user.send(match[1])
        else
            render :nothing => true, :status => 500
        end
    else
        raise NoMethodError, "undefined method '#{name}'"
    end
  end

This is a good time to point out that care must be taken with the choice of attributes=, update_attribute or update_attributes. attributes= is a good choice, because it pays attention to protect attributes that have been excluded from mass-assignment. Using attr_protected or attr_accessible gives one a consistent mechanism to protect sensitive attributes, which one might assume a user object has. attributes= requires an explicit save call on the object afterwards. update_attribute is probably never a good choice, since it bypasses validations. update_attributes actually just calls attributes= and does the save in one call, so it conforms to the protections provided by attr_protected and attr_accessible. update_attributes is probably the best choice here.

The best solution is to use the attr_protected or attr_accessible mechanisms in your model to restrict sensitive attributes from mass assignment. Since the following reference code uses update_attributes, your attribute restrictions will apply to this update mechanism:


  def method_missing(name)
    match = /\Aupdate_([a-z]{1,20})\Z/.match(name.id2name)
    if match && current_user && current_user.attributes.keys.include?(match[1])
        if params[:value]
            current_user.update_attributes(match[1] => params[:value])
            current_user.reload
            render :text => current_user.send(match[1])
        else
            render :nothing => true, :status => 500
        end
    else
        raise NoMethodError, "undefined method '#{name}'"
    end
  end

This code is simplified, since the object we need to reference is readily available in the session hash. If you need to pass an object id, you would need to extend the AJAX call (using the :with option for the JavaScriptMacrosHelpers) to include the id as an additional parameter and retrieve the object in the normal way.

Notice this code uses update_attributes and constructs a hash of the attribute name and the value.

In the view, you would have erb such as:


Name: <p id="user_name"><%= h @current_user.name %></p>
<%= in_place_editor 'user_name', {:url => '/user/update_name' } %>

If you feel you need to deviate from the normal attribute protection scheme (you should not as a general principle), then you can do something like this in your model:


  def self.protected_attributes
    ['id','type','salt','hashed_password','verify_code']
  end

  def safe_attribute?(attr_name)
    (self.attributes.keys - User.protected_attributes).include? attr_name
  end

and then add it to your method_missing hook like:


  def method_missing(name)
    match = /\Aupdate_([a-z]{1,20})\Z/.match(name.id2name)
    if match&& current_user && current_user.safe_attribute?(match[1])
        if params[:value]
            current_user.update_attributes(match[1] => params[:value])
            current_user.reload
            render :text => current_user.send(match[1])
        else
            render :nothing => true, :status => 500
        end
    else
        raise NoMethodError, "undefined method '#{name}'"
    end
  end

Hooking method_missing is a little dirty. You might actually need to hook method_missing for other (better) reasons. Here is a solution that is dirty in a different way. In the controller:


private
 
  def called_as
    /`([^']*)'/.match(caller[0])[1]
  end

public

  def update_attr_generic
    match = /\Aupdate_([a-z]{1,20})\Z/.match(called_as)
    if match && current_user && current_user.attributes.keys.include?(match[1])
        if params[:value]
            current_user.update_attributes(match[1] => params[:value])
            current_user.reload
            render :text => current_user.send(match[1])
        else
            render :nothing => true, :status => 500
        end
    else
        render :nothing => true, :status => 500
    end
  end

  alias_method(update_name, update_attr_generic)
  alias_method(update_phone, update_attr_generic)

The obvious down side being that you have to create an aliased method name for each attribute you want the ability to update (code generation?). I am jumping through this naming hoop because, by default, the AJAX stuff included in rails likes to post a single parameter (:value) to a named action (commonly :controller/:action). If you modify the AJAX call to send the attribute name and value, or have a more complicated route map and URI, then you can ditch the method name introspection, regular expression and alias_method foolishness and have a single update_attr method. For example (replace ObjModelClass with the model name):


  def update_attr
    obj = ObjModelClass.find(params[:id])
    if obj && obj.attributes.keys.include?(params[:attrname])
        if params[:value]
            obj.update_attributes(params[:attrname] => params[:value])
            render :text => obj.send(params[:attrname])
        else
            render :nothing => true, :status => 500
        end
    else
        render :nothing => true, :status => 500
    end
  end

There you go! A generic attribute updater for a given AR model that adheres to attribute protections placed with attr_protected or attr_accessible. However, you will have to do a little more work in your views to make the AJAX methods pass :id, :attrname and :value parameters to the action. I like to simplify my views, so I often use the dirty, lazy method_missing implementation previously described.

BTW, these methods respond with the value of the attribute after the update, successful or not. If no value is passed in the parameters, the response is an HTTP error code, which most Javascript AJAX libraries handle OK.

Another quick point is that this is pretty generic functionality. All the attribute validation and restrictions should be in your models. Models are the object-oriented part of the RoR MVC implementation. Controllers are the procedural part of your application. Keep your controller code simple. If you find yourself duplicating code in controller actions, it probably can be put into a model (one place to fix and DRY). By using update_attributes or attributes=, you leave the security and enforcement in the model where it belongs.

Go ahead, pick your poison.

Speaking of regex and ruby...

2007-01-31T23:28:00.000-08:00

I came across this link: http://macromates.com/textmate/manual/regular_expressions

This is a nice summary (with link to original source document) of the regex library used in ruby. Related the UTF-8 regex issues I commented on, the document describes what subset \w actually matches with u extension. It may be out of date (not sure), but it is the most thorough documentation of the ruby regex I have stumbled across.

Ugly bug in prototype/rails

2007-01-31T17:38:00.000-08:00

While testing an app, I just noticed that the observe_field (Form.Element.EventObserver), defined this way:


new Form.Element.EventObserver('customer_Name', function(element, value) {new Ajax.Request('/foo/validate_field', {asynchronous:true, evalScripts:true, parameters:'customer[Name]=' + value})})

passed the value through without any munging, which is then parsed by rails into the params hash. Rails param parsing simply splits on & and seems to take the last assignment, so input something like '&' into an input tag with the name 'customer[Name]' will generate:


Processing FooController#validate_field (for 127.0.0.1 at 2007-01-31 17:58:39) [POST]
  Session ID: 02685db5b8bfde88f7b858eeec0b3a9d
  Parameters: {"action"=>"validate_field", "amp;"=>"", "controller"=>"foo", "customer"=>{"Name"=>""}}
  Customer Columns (0.001306)   SHOW FIELDS FROM customers
Completed in 0.13547 (7 reqs/sec) | Rendering: 0.02115 (15%) | DB: 0.00131 (0%) | 200 OK [http://127.0.0.1/foo/validate_field]

And input '&customer[Name]=bar' will generate a request like:


Processing FooController#validate_field (for 127.0.0.1 at 2007-01-31 18:16:20) [POST]
  Session ID: 02685db5b8bfde88f7b858eeec0b3a9d
  Parameters: {"action"=>"validate_field", "controller"=>"foo", "customer"=>{"Name"=>"bar"}}
  Customer Columns (0.001054)   SHOW FIELDS FROM customers
Completed in 0.13539 (7 reqs/sec) | Rendering: 0.02108 (15%) | DB: 0.00105 (0%) | 200 OK [http://127.0.0.1/foo/validate_field]

Oops. That is not good. Considering the parameters you hope for would include '"customer"=>{"Name"=>"&customer[Name]=bar"}, but instead you get "customer"=>{"Name"=>"bar"}. The value from the field, when parsed by rails, over-wrote the previously generated customer hash. This has a number of implications:

- if you are using an AJAX (prototype) methods to validate form field data, input that includes ampersand(s) will have some or all of the string hidden from the receiver. This assumes that the controller method is trying to receive the data to validate from a specific key(s) in the params hash.

- if you are using AJAX (prototype) methods to receive model-mapped input to update an object, it is possible for someone to piggyback additional attributes (assuming bulk attribute assignment) straight from input fields. No mucking around and hand-crafting a request. Just &model[current_attribute]=data&model[other_attribute]=data_you_did_not_expect. Of course it is still subject to validation and accessible restrictions (you do use those right?). This is not the default BTW. Most of the helpers like observe_field simply send the data from the field as the post body. It will get parsed, and ampersands will cause separate entries in params. It really depends on your expectations and whether you use the :with option to place the data where you expect to find it (or mapped for attribute assignment to an AR object).

- worse I assume. It really depends on a lot of factors. If your models have all attributes accessible to bulk assignment, you use one or more attributes to store information that is state sensitive and you update attributes on that model using AJAX, you may find yourself in trouble.

There are a couple ways to code around this issue, but the right one for you depends greatly on your implementation.

UTF-8, rails vaildations and all that jazz

2006-12-15T16:38:00.000-08:00

To be honest, it never occurred to me that Ruby did not natively support Unicode given its roots. Alas, Ruby really has no clue about multi-byte character encodings, except Regexp that can handle UTF-8 if the 'u' specifier is used. But wait, given that UTF-8 is the default encoding for XHTML etc., what affect does this have in Rails applications? A lot and not much at all at the same time. Let me explain the little I know about this.

I ran into this issue on the first rails app I deployed. Even though I had pretty tight validation regexs on my models, every once in a while user provided content would clearly contain Unicode and not display correctly. Damn, that is scary. My validations where not stopping this data from making it into the model. The app served out XHTML 1.0 Strict pages with charaset set to UTF-8. By default browsers POST form data in the same encoding as the page containing the form; In this case UTF-8. All it takes is one cut&paste from MS Office into a form field (think bullet points etc.) to end up with Unicode in the form parameter values. The reason I was ending up with multi-byte values that were not displaying correctly it two-part.

The more trivial side of the issue is that it turns out Safari has a bug when handling response data from XmlHttpRequest that contains multi-byte encoded characters. This issue is resolved by explicitly setting the encoding on AJAX responses. The more troublesome issue is how the multi-byte characters made it past the validations on the ActiveRecord models. After a little experimentation I came up with this:


irb(main):036:0> utf8 = "\342\200\271script\342\200\272"
=> "\342\200\271script\342\200\272"
irb(main):037:0> test_regex = /^[\w]+$/u
=> /^[\w]+$/u
irb(main):038:0> m = test_regex.match utf8
=> #
irb(main):039:0> m[0]
=> "\342\200\271script\342\200\272"
irb(main):040:0> test_regex = /^[-a-zA-Z0-9_.]+$/u
=> /^[-a-zA-Z0-9_.]+$/u
irb(main):041:0> m = test_regex.match utf8
=> nil

Argh, \w in UTF-8 enabled Regexp matches most, if not all, of the UTF-8 character set above the single-byte encodable ASCII subset. Fortunately, it appears that Ruby's Regexp UTF-8 string handling does not honor overlong-encoded characters, so it appears if you are doing validations that exclude many of the characters generally needed to inject HTML or Javascript, it is unlikely that they can simply be avoided by using overlong UTF-8 encodings. That is good news. However, if multi-byte encoded characters make it into your models and you use methods like sanitize() or h() to do some safety filtering on presentation, don't expect them to do their jobs well.

In summary, there is potential for some security issues to arise out of acceptance of multi-byte encoded data, but the most straight-forward issues like direct tag and script injection will generally not escape past your input validations. As I research this more, I will keep you posted.

Session fixation in Rails

2006-12-14T22:03:00.000-08:00

If you are not familar with session fixation vulnerabilities, just google it or hit up wikipedia.

By default Rails supports sessions and uses CGI::Session to handle session management. CGI::Session defaults to setting the session id in a cookie, but it has a backup:

"If the client has cookies disabled, the session id must be included as a parameter of all requests sent by the client to the server. The CGI::Session class in conjunction with the CGI class will transparently add the session id as a hidden input field to all forms generated using the CGI.form HTML generation method. No built-in support is provided for other mechanisms, such as URL re-writing."

OK, so there is an avenue to exploit session fixation without having to be able to manipulate cookies (through a client-side vulnerability) in Rails. Hack a little on CGI.form to figure out the parameter name that will make it accept a session id from a form parameter (so the attack would be to modify the form served via an XSS vulnerability or rehost the form in a phish variation). Sessions are on by default in Rails, so we need a to remediate this issue. Here are a couple of options:

1. Disable session support in your application if you can. It may be a little more of a design challenge, but in some cases you can safely avoid having to maintain state between client and server. Edit app/controllers/application.rb and add "session :off" to it. There you go, sessions are disabled.

2. OK, you need sessions. Fine. In an app I wrote, there are a few views that are unauthenticated, a bunch of views that are authenticated, a login mechanism and sessions are used to store error/response messages in the unauthenticated views and to hold the user object in the authenticated views. The risk of session fixation in the unauthenticated views in very minimal, but it does give an attacker a very easy mechanism to obtain a valid session id. Once the user is authenticated, knowing the session id provides access as the user, but that is a different issue (session hijacking) assuming we fix session fixation. We really just need to protect the transition for unauthenticated to authenticated by changing the session id on the transition, which is controlled by the login mechanism. The application is largely RESTful, so a GET to /user/login yields the XHTML login form and a POST to /user/login attempts to authenticate the user with the parameter values. In the user controller, I added "session :new_session => true, :session_expires => (Time::new + (60 * 60 * 24)), :only => :login, :if => Proc::new {|req| req.post? }" The :if parameter provides an anonymous function that checks whether the current request is a POST to be consistent with the REST semantics of user controller actions. The other parameters are self-explanatory. Even if an attacker steers the user to the login view with a know session id, when the user authenticates the session id will change. There you go, authentication yields a different session id.

For the auditors out there, you should be looking for "session :off" or "session :new_session => true" that covers the unauthenticated/authenticated transition in those rails apps growing like weeds in your front yard.

Oh, and I am sure there is a more elegant solution, and with the beauty of rails, I am sure someone will figure out how to package it in a plugin. I also use one-time, time-limited nonces to validate forms were served from the server, but that only hinders the form rehosting (phish) version of the attack (forces the attacker to proxy, a different threat altogether, or periodically re-GET the form, which is generally easy to spot in logs). Dealing with the session fixation issue is still necessary.

Welcome to the New-school

2006-12-14T22:02:00.001-08:00

I finally decided to contribute something to the global knowledge-base. Whether this something is of interest to you is yet to be determined. You may ask what is the deal with the blog name, so let's start there.

In the underground, hackers were often referred to as either old-school or new-school, with the latter term being somewhat derogatory. I always considered myself on the cusp. The old-school was long in the tooth when I started hacking freebsd, linux and Windows NT 3.1. Shell accounts on Netcom did not compare to X.25. But I had the old-school ethic and interest, and many of my friends are straight-up old-school: the NWA of globally connected networks. WTF you say? I am getting there--just explaining my relationship with the new-school term.

The role of a security engineer is rapidly changing. Five to ten years ago, security engineers handled a lot of firewall configuration; IDS deployment; anti-virus; pentesting and in enlightened environments, code review/programmer training. Now, security engineering is moving towards an aspect of, or integral part of, the software development process. I think the following factors are contributing to this change:

1. For a number of very pragmatic reasons (simplifying operations and responsibility, improved product knowledge, easier to configure products with better defaults, device-ifying of many things), much of the network side of security engineering has moved into network engineering. Any product company building a networking device with security functionality should be expecting it to be managed by network engineers. Network engineers are, in the end, responsible for the network infrastructure, and if packets flow through it, it is part of the network infrastructure. Having split-responsibility for networking devices complicates trouble-shooting and increases MTTR.

2. In-house software development is rapidly increasing as many business leverage the internet for some or all of their revenue stream. I worked at Amazon.com for a total of three years, and it may come as a shock to hear that Amazon largely considers itself a software platform development company. Simply put, Amazon's software platform enables its revenue stream, and Amazon's ability to be agile with regard to features and platform direction is critical to its success. Amazon is an internet company, but not only internet companies are highly dependent on software agility. The financial industry stands on the cutting edge of computing and software development. And software product companies have taken notice; look at Microsoft. What does this have to do with security engineering? If you have been paying any attention to the security space, you have witnessed the rise of the application-level vulnerability. Core OS network-connected services where once the focus of vulnerability research. If one thing was learned about security, it was that only the necessary services should be exposed externally. Indeed, the average company has a radically improved external security posture with regard to exposed services. What is left? Web applications springing up like weeds and the SOA offerings coming online. Of course there is also the core software these applications depend on, going all the way down to the OS (and beyond).

Where should the focus of security expertise be applied? It does not take a genius to figure that it is better to build security in than try to bolt it on, hence the re-focus of security engineering towards the software development process. This brings us to the new new-school.

Some of the old-school are not all that prepared for the new new-school, while others are deeply thankful to finally start working on the real security problem: design and implementation of software, hardware and everything inbetween. Design and implementation baby. Yeah, now we are talking. Well, sort of. I also have another little bias that is a love of dynamic programming languages. To some of the old-school, dynamic programming languages are considered toy-like, suitable for scripting at best, near sacrilegious, dangerous and should be exercised from real programs. Not in this new new-school; we are all over this dynamic programming language sh*t and intend to use it for everything we can get away with.

There you go. This blog is about security as part of software design and implementation with a bent on dynamic programming languages. It actually just might be a place that I jot down little things I notice and learn while hacking Ruby on Rails apps, but I don't know if that is enough for a BLOG, so I will pretend the scope is big and worthy of notice. Time for me to shut up and add some real content.