Writing Secure Code

Security is the paramount issue when deploying a product over the internet. Here I am trying to collect a set of principles for writing secure code, that I have collected from different places on the internet.

1. Any functionality you have built can be used by anybody with an ulterior motive. Design functionality/endpoint/service/microservice with a mindset that guards against the most rogue user of that functionality.
2. All user input coming your way should be assumed to be coming from a most rogue user of the functionality. Follow the steps defined below. 
1. Sanitize
2. Validate
3. Execute
4. Display feedback
3. If you have defined endpoints and provided a web client or an app, Don't assume that is the only method that will be used to access your endpoints.
4. API Keys are as important, if not more important, than usernames and password. Guard them like that.
5. Any place you are using API keys or passwords, think, what will you do in case these are exposed. How will you handle key rotation? Particularly if you have a device as your client, it should be capable of handling key rotations, or forced password change.
6. Passwords and API keys should not be part of the code and committed to configuration management systems.
7. If you are using encryption with keys (e.g. AES 256), think where will be store the keys themselves. Think of Hardware Security Modules.
8. Unless your occupation is a security researcher, don't fall to the temptation of designing your own encryption algorithm. Security by obfuscation is just a bad idea.
9. Before storing any data, think about what you intend to do with it. Any customer data stored in your systems have potential to be leaked. Don't store a piece of data that you have no need for.
10. Pay attention to filtering rules for your logging and audit your logs for any accidental sensitive data reaching the logs.
11. Heed warning from compilers, lint, and other analysis tools. A sprint is done only when all the warnings have been removed.
12. In this time and age, don't use HTTP, Stick to HTTPS with at least TLS/1.2
13. Have a policy on employee turnover. Many times ex-employees may be the biggest source of data leakage. 
14. Specifically design and test against injection risks. Minimize native SQL queries as much as possible.
15. Specifically design and test against broken authentication and authorization. Many systems suffer from issues such as users assuming identities of other users, users able to authorize themselves for higher roles etc. Take the help of a security researcher if you don't have staff on the team.
16. Each piece of data collected and stored should be specifically annotated with the level of privacy and encryption required. Detailed thought related to the type of data needs to be given right at the design stage.
17. Auditing sensitive data is mandatory. The system needs to have an audit system built that can help us retrieve sufficient breadcrumbs in case of a customer complaint about data compromise.

These are some of the items that I could think of. Comment with your thoughts.

What are some fundamentals of security every developer should understand?
OWASP Top 10 - 2017

Why I finally believe Microsoft is a changed company

The media has been talking about how Microsoft is a changed company. I never believed that story. I have seen many large companies trying to change and eventually fail. There are examples of Motorola, Bell Labs, HP, Nokia; all of them trying to change but failing at it.

Yesterday one of my friends introduced me to this new app called "SMS Organizer". The name could have been better but leaving that part aside, it looks like a typical thing that you expect Googles and Apples of the world to do, not expected from Microsoft at all. The fact that they built an app that solves a key problem and released it on android indicates to the fact that it is a changed company.

The app itself is fantastic, it takes a "Inbox by Gmail" approach to the SMS. I believe this is something that Google should have done years ago but they chose to ignore it. Following are the key features of the app.

• You have Bill reminders right on top
• Messages are categorized as personal, promotional, transactional. 
• You can block SMS messages. 
• You can put rules to delete messages like OTP and promotions to be deleted after a certain period. 
• You can backup your SMS messages to Google Drive

It is a great app, I am still playing with it to see if it completely replaces the native messaging app but as of now, it looks promising.

Why most organizations are wrong about AI

Everybody has joined the AI or ML bandwagon. The corridor talk seems to be about words like Deep Learning, CNN, Tensor Flow and other related jargons. As companies are devoting more and more resources into machine learning, the results don't seem to be matching up the investment.
The biggest reason behind that seems to be the fact that the machine learning is not really about the underlying algorithm, but it is about the data in a form that can be used.
For any business to use the data that they are collecting for their ML solutions, they first need to define what inferences they want to draw. Once they know what inferences they want to draw, they need to look at the data and decide whether those inferences can be drawn from the data they collected. If the answer to that question is yes, an effort needs to be started to accurately tag the data and define the training set for their data.
All this is a lot of work and even with this the end result may not be worth the investment. Organizations need to understand, ML and AI is more about the data that they collect and less about that fancy algorithm that you can cook up.

Using google blobstore through android client

Recently I have been working on an Android project that required me to store some images into cloud. Upon reading through google documentation, I learnt that the best place to store dynamic images is Google BlobService. Unfortunately for me it required some iterations to figure out how to use this service. So I thought I would document it here for the help of others.

It is important to understand how the BlobService workflow works. Following is the typical workflow of a blobservice.

• The client first creates an upload URL where the image would be uploaded. Since it is hard (not possible) to do a decent authentication scheme around blobservice, I decided to use a appengine endpoint to create the upload URL for me. The code for appengine endpoint implementation is very simple.

    @ApiMethod(name = "getUploadURL")

    public UploadURL getUploadURL() {

        BlobstoreService blobstoreService = BlobstoreServiceFactory.getBlobstoreService();

        return new UploadURL(blobstoreService.createUploadUrl("/_ah/uploads"));

    }

The arugument to createUploadUrl is the suffix that would be added to your appengine base URL. We need to understand that the actual  image upload is unauthenticated, i.e. anybody who has the URL can upload the image but the endpoint itself is authenticated and hence nobody would know the URL to upload anything except authenticated users. 

• Now that we have the URL, the first thing that we need to do is to convert the image into a byte array that we can upload. I have created following utility function to do just that. Basically I don't want to store full sized images, I have created this function that takes a Bitmap and scales the image and converts it to a byte array. The images is scaled with actual aspect ratio with the width fixed at maximum PRODUCT_IMAGE_WIDTH.

    public static byte[] getByteArrayFromBitmap(Bitmap bitmap, boolean scale) {

        Bitmap scaledBitmap = bitmap;

        if (scale) {

            scaledBitmap = scale(bitmap);

        }

        ByteArrayOutputStream stream = new ByteArrayOutputStream();

        scaledBitmap.compress(Bitmap.CompressFormat.PNG, 100, stream);

        return stream.toByteArray();

    }

    public static  Bitmap scale (Bitmap source){

        int w = source.getWidth();

        double factor = 1.0;

        if (w > PRODUCT_IMAGE_WIDTH) {

            factor = ((double)w) / ((double)PRODUCT_IMAGE_WIDTH);

        }

        else {

        }

        int dw = new Double((((double)factor) * ((double)(source.getWidth())))).intValue();

        int dh = new Double((((double)factor) * ((double)(source.getHeight())))).intValue();

        return Bitmap.createScaledBitmap(source, dw, dh, false);

    }

• Now that we have the image as a byte array, we upload the image from the client using this URL. Since google has removed the old HTTPClient, I had to figure out how to use OkHTTP which seems to be the new HTTP client that needs to be used. It was really not very hard to do this, it is just that documentation around OKHttp is very sparse and it takes some trial and error to make this work. We set a response header so that the client can receive the cloud key that is generated by the blob service. We need to know this key to retrieve the image later.

            String uploadURL = api.getUploadURL().execute().getUrl();

            OkHttpClient httpClient = new OkHttpClient();

            String filename = UUID.randomUUID().toString() + ".png";

            RequestBody body = new MultipartBuilder("image-part-name")

                    .type(MultipartBuilder.FORM)

                    .addFormDataPart("file", filename,          

            RequestBody.create(MediaType.parse("image/png"), imageByteArray))

                    .build();

            Request request = new Request.Builder().url(uploadURL).post(body).build();

            Response response = httpClient.newCall(request).execute();

            String cloudKey = response.header("X-MyApplication-Blob-Cloud-key");

            // We can store this cloud key in cloud store entity with the other data related to 

            // image so that we can retrieve it later.

• Let's look at the implementation of blobservice. We basically need to create a servlet to handle the upload request. The doPost of the servlet should handle the upload. WIth some trial and error, I figured out that the blobservice API looks for a multipart payload with image byte array and header with the name "file". You can also provide the bytearray type, otherwise it decodes it with the filename extension that you provide as part of the value.

    @Override

    public void doPost(HttpServletRequest req, HttpServletResponse res)

            throws ServletException, IOException {

        Map> blobs = blobstoreService.getUploads(req);

        List blobKeys = blobs.get("file");

        if (blobKeys == null || blobKeys.isEmpty()) {

        } else {

            res.setHeader("X-MyApplication-Blob-Cloud-key", blobKeys.get(0).getKeyString());

        }

    }

• Retrieving the image is also very simple. You need another servlet to do that. In my case I have a servlet with a GET method which returns the image.

    @Override

    public void doGet(HttpServletRequest req, HttpServletResponse res)

            throws IOException {

        BlobKey blobKey = new BlobKey(req.getParameter("blob-key"));

        blobstoreService.serve(blobKey, res);

    }

As we can see, it is quite easy to build a work flow where the image is stored with blobservice and other metadata is stored with google cloud store with cloud key. The image can be retrieved based on the other metadata and lookup from cloud store.

Why I think Java is bad for computer industry

I have felt this for quite a long time and after multiple events that have happened in past, I have come to the conclusion that java should not be the first language of anybody who intends to learn programming. Even the regular java programmer should take their time off and code in some other language once in a while.

Why am I saying this. In last many years, I have run a programming competition for the employees of the companies where I have been working in. These are individuals with significant experience in programming. We generally allow people to code in C/C++ and Java.

Most of the problems that we design for these competitions have a standard statement written in them. 

The input is read from standard input till EOF and the output should be written to standard output.

Almost every year I am asked clarifications related to this statement in many different forms.

1. My program reads from a file standard-input.txt and writes to standard-output.txt
2. My program reads one line at a time and then you have to run it again 
3. Standard input doesn't have EOF

Almost always these clarifications come from experienced Java programmers. This leads me to believe that people who start programming in languages that have extremely rich set of libraries forget the basic constructs of language, programming and operating system. These are some basic constructs that I would expect everybody would know Even otherwise, it is just a simple google query away. Since most of these people are experienced programmer, their presumption is that the question must have a typo and then don't bother looking it up.

There, it is off my chest now. I can breathe properly.