Blog on Osman Goni Nahid

We were hashing every byte twice: making 500 GiB volume imports fast

Sat, 19 Sep 2026 00:00:00 +0400

On the platform I work on, a workspace volume is a Kubernetes PVC that users fill from Hugging Face or an S3 bucket and then mount into training or inference workloads. The import runs as a one-off pod: download, then record metadata (a file tree with per-file hashes, and one digest for the whole volume) back to the control plane.

Customers importing 500 GiB+ models started reporting that the download finished in a reasonable time and then the pod sat for hours in the metadata step. This is the story of why, and what fixed it.

Common Problem on Scaling Nodejs

Tue, 22 Jan 2019 00:00:00 +0400

I have built lots of NodeJS apps and scaled them to millions of requests. The most common problem you will see is Garbage collection pauses and memory leaks which sometime gives random distribution of time response of requests. The problem is not the GC but the way you have written code.

Another is Memory leaks you will find them in production and see your app crashing a lot and no idea why it crashed because locally everything seemed to be fine but you gotta find them.

Database Indexing, performance and How It Works

Sat, 08 Dec 2018 00:00:00 +0400

Indexing in Database is a wide topic. Database indexing plays a important role in your query result performance. But like everything this too has a trade off. I know and assumed that you already worked with Indexing and you indexed your few field of your database table, But in this post I will explain a bit tho I’m not a good writer will try to finish within short. What is Indexing, How it works etc.

No more express-generator on API building with NodeJS

Mon, 19 Mar 2018 00:00:00 +0400

Combined (Front-end and Back-end) application is no more nowadays. The way of doing front-end and backend in same application with same stack is old now. The problem with this model is that it tightly couples front-end and back-end development into a single development stack and imposes restraints on technology choices.

But whilst this has been the best solution for some time, this was primarily due to poor performing web browsers not being able to handle complex operations that are required to run a modern day website.

Resolving Google Play services Incompatibility in React-Native(Android)

Thu, 21 Dec 2017 00:00:00 +0400

In react-native when we use any external library where play-services are required like react-native-maps for google map there is an android project into this library where play-services v8.x.x included into gradle and in your project you are using play-services v9.x.x for another reason

We are using react-native for our mobile application where have to build and deploy both android and ios from same branch. Last week our frontend (mobile) team were facing an issue with play-services version incompatibility and multidex issue. That MultiDex is a common issue in native android for different version of google-play-services and we can solve it by two way, either we can make a common version for all types play-services dependencies or in gradle we can set multidex : true. So it’s easy for native.

Use Cases for Using Let and Const in JavaScript

Wed, 20 Dec 2017 00:00:00 +0400

There are now two new ways to declare variables in JavaScript: let and const.The only way to declare a variable in JavaScript was to use the keyword var. To understand why let and const were added, it’s probably best to look at an example of when using var can get us into trouble.

function getThing(isDoes) {
 if (isCold) {
 var foo = 'Grab a jacket!';
 } else {
 var bar = 'It’s a shorts kind of day.';
 console.log(foo);
 }
}

let and const

Variables declared with let and const eliminate this specific issue of hoisting because they’re scoped to the block, not to the function. Previously, when you used var, variables were either scoped globally or locally to an entire function scope.

Separating Schema, Method and Statics in Mongoose NodeJS

Fri, 01 Dec 2017 00:00:00 +0400

Database logic should be encapsulated within the data model. Mongoose provides 2 ways of doing this, methods and statics. Methods adds an instance method to documents whereas Statics adds static “class” methods to the Models itself. like example below:

const bookSchema = mongoose.Schema({
 title: {
 type : String,
 required : [true, 'Book name required']
 },
 publisher : {
 type : String,
 required : [true, 'Publisher name required']
 },
 thumbnail : {
 type : String
 }
 type : {
 type : String
 },
 hasAward : {
 type : Boolean
 }
});

//method
bookSchema.methods.findByType = function (callback) {
 return this.model('Book').find({ type: this.type }, callback);
};

// statics
bookSchema.statics.findBooksWithAward = function (callback) {
 Book.find({ hasAward: true }, callback);
};

const Book = mongoose.model('Book', bookSchema);
export default Book;

It’s awesome right? Yeah!. But what if, you have a complex schema , large amount of field and schemas the number of methods and statics are pretty large.

One-Way vs. Two-Way Data Binding

Sat, 25 Nov 2017 00:00:00 +0400

Two-way binding is simply a way to sync data to a view and vice versa (mvvm pattern). One-way data binding refers to the design pattern found in component based frameworks, where a child component cannot modify data passed by a parent component, as that creates spaghetti code.

Image source slideshare

One-Way Data Binding

One-way data binding refers to the design pattern found in component based frameworks, where a child component cannot modify data passed by a parent component, as that creates spaghetti code.

Calling and Executing C++ Code from Node.js

Sun, 19 Nov 2017 00:00:00 +0400

There is three general ways of integrating C++ code with a Node.js application - although there are lots of variations within each category like: Automation, Shared Library and Addon. Each of them has advantage and disadvantage as well

It’s not that hard to execute C++ code with JavaScript. NodeJS can not only load and/or execute the JavaScript libraries and/or files, but also be extended with native modules (compiled C/C++ code). While this does not mean that you should wipe out your existing JavaScript modules into C++, this knowledge might come in handy in specific use cases.

Working with Geospatial support in MongoDB with NodeJS

Sun, 19 Nov 2017 00:00:00 +0400

MongoDB’s geospatial indexing allows you to efficiently execute spatial queries on a collection that contains geospatial shapes and points. This tutorial will briefly introduce the concepts of geospatial indexes, and then demonstrate their use with $geoWithin, $geoIntersects, and geoNear.

At my current project I’m working on requires storage of and queries on Geospatial data. We are using MongoDB, which has good support for Geospatial data, at least good enough for my needs. This post walks through the basics of how to work with mongodb geospatial data in Nodejs application. We are using MongoDB 3.4. Mongo has some very awesome and useful function for doing spatial operation like $geoNear $near $nearSphere $centerSphere etc. All of them has some restriction like they don’t support shading. Here I will use $near with nodejs example.

When Object.assign() Is Risky

Tue, 14 Nov 2017 00:00:00 +0400

Javascript Object.assign allows you to copy one set of an object non-inherited properties to another.It has a signature of Object.assign(target, …sources). The target object is the first parameter and is also used as the return value. Object.assign() is useful for merging objects or cloning them shallowly. code sample below :

const adminPrivileges = {
 canCreate: true,
 canDelete: true,
 canUpdate : true,
 canRead : true
};

const userPrivileges = Object.assign({}, adminPrivileges, {
 canCreate: false,
 canDelete: false,
 canUpdate : false,
 canRead : true
});

console.log(userPrivileges);
// { canCreate: false,
// canDelete: false,
// canCreate: false,
// canRead: false }

But The Risky is that if you forget to include {} will result in accidentally modifying the original object causing nasty consequences later and its hard to notice.

ES6 Style Module Import and Destructuring Assignment

Thu, 09 Nov 2017 00:00:00 +0400

Though I’m a big fan of ES6 and always I try to do everything with ES6 instead of ES5 but sometimes have to write JS code into ES5. That’s why I try to make the ES6 flavour into ES5 codes.

In JavaScript it is often more clear to only import parts of modules/objects when they are needed. Before ES6 this was a cumbersome task which required assigning to multiple variables. For example, in Node.js:

Mongo DB change stream new Feature of Mongo 3.6

Wed, 08 Nov 2017 00:00:00 +0400

Those days are gone when people didn’t expect information for momentary access to data. Nowadays expectation everywhere is real-time action and result. Something is happened? not 1 day ago, not even 1 min ago but just now. Being notified constantly on any change of data points a bit critical from many application nowadays. So the upcoming MongoDB3.6 introduces a new $changeStream aggregation pipeline operator.

Change Stream:

Change streams are being implemented in the driver with a new aggregation operator, $changeStream and watch method in the API. Where able to specify a $change stage at the beginning of our pipeline and request that notifications are sent for specific changes or hooking something to a particular collection.

Significant Benefits of Microservice Architecture Pattern

Wed, 08 Nov 2017 00:00:00 +0400

In Microservice architecture there is a number of benefits. I’ll point some major benefits of microservice rather than monolithic approach, what I got from my experience.

Splitting Problem Complexity:

Microservice architecture pattern tackles the problem complexity by decomposing what would otherwise be a monstrous monolithic application into a set of services. While the total amount of functionality is unchanged, the application has been broken up into manageable chunks or services. Each service has a well-defined boundary in the form of a remote procedure call driven message API(RPC). The microservice architecture pattern enforces a level of modularity that in practice is extremely difficult to achieve with a monolithic code base. We/Developers are able to focus on a specific problem or part of the problem on a specific boundary of a service. On the other hand, Individual services are much faster to develop and easier to understand and maintain.

Flavour of Async/await ES2017

Wed, 01 Nov 2017 00:00:00 +0400

One really exciting thing for JS Developer that last week was dropped Node 7.6 which brought support for async/await. If you haven’t heard of async/await before, it’s an amazing update to JavaScript (ES2017) that allows us to write synchronous looking asynchronous code without nested callbacks or even chained .then() promises.

/* Forgot Password Flow*/
export default forgotPassword = async(req,res,next) => {
 // Checking if user exist
 const user = await User.findOne({email : req.body.email}) ;

 if(!user){
 req.flash = ('error' , 'No Account found with this email');
 return res.redirect('/login')
 }
// Set password reset token for user
 user.resetPasswordToken = crypto.randomBytes(20).toString('hex');
 user.resetPasswordExpires = Date.now() + 3600000 ; // 1 hour
 await user.save()
// Sent Password reset email to User
 await mail.send({
 user,
 file:'password-reset',
 resetUrl : `http://${req.headers.host}/account/reset/${user.resetPasswordToken}`,
 subject: 'Password Reset'
 });
req.flash('success', 'You have been emailed a password reset link.');
res.redirect('/login');
};

Happy coding :)

A Fundamental Go-through into Javascript World

Tue, 08 Aug 2017 00:00:00 +0400

JavaScript is a cross-platform, object-oriented scripting language. It is a small and lightweight language. Inside a host environment like a web browser.

JavaScript contains a standard library of objects like other languages, such as Array, Date, Math and a core set of language elements such as operators, control structures, and statements. Core JavaScript can be extended for a variety of purposes by supplementing it with additional objects.

This is a fundamental overview to introduce new developers to JavaScript and help experienced developers learn and explore more about its best practices.