Archive for category Non-Technical
A neutral perspective of the current database industry
According to db-engines, there are more than 317 database software in the market right now. If you have requirements to develop a new application , how do you know which database to use? When I started my career in 2011, the de facto standard was to use Oracle database. It was mature, reliable and able to support multiple types of workload. If you can’t afford Oracle database, you could use MySQL, if you require high performance with HA capabilities. If you require a mature product with strict compliant to ANSI SQL standards, there’s always PostgreSQL. However, Oracle is still the #1 choice for a lot of reasons. Also, Oracle DBAs were also paid the most, compared to other databases DBA.
Five years has passed, a lot has changed in the database industry. Even though Oracle is still the #1 in the market, there are many other comparable choices now. Oracle supporters will claim that Oracle database is still the best and no other database products will come close. They will also claim that NoSQL products are overhyped and will die down soon enough. Open-source supporters will spread hate on Oracle, claim that it’s overpriced and that you don’t need Oracle database at all.
In my opinion, enterprise companies AKA companies whom are loaded with cash, will still continue to pay 7 figures for Oracle support contract. They continue to do so because they are not willing to migrate their legacy application to something else. Also, It doesn’t make sense for them to use other database software when they have unlimited license usage for Oracle software. It’s also possible that the CIO aren’t willing to risk his/her neck by changing the database stack 🙂 For companies that aren’t as rich, usually their ERP software will still be stuck using Oracle database. That’s if they haven’t move to a cloud ERP yet. For new applications, they are already looking at alternatives.
Oracle is still the #1 database, in terms of technical superiority. No other databases vendors have implemented Oracle’s Real Application Cluster (RAC). It’s an extremely powerful feature that offers possibly, zero downtime for node failure. However, how many real life applications require that? Is it feasible to have that feature for an additional 23,000 USD/core? Is it acceptable to have 15s of downtime during node failure instead? That would decrease your cost to about nothing for licensing.
If you have a multiple terabyte application, the cost of having that on a Oracle database is not going to be cheap. A four-core EE licensed server is going to cost you at least 200,000 USD already. If you run it on MongoDB or MySQL, it’s not going to cost you anything. Obviously, your application needs to be able to able to support that (ACID compliant). If you help your company to save that 200k USD, you might find a sweeter bonus at the end of the year!
Oracle database has got a lot of great technology behind it. The database optimisation engine is far more impressive than any other vendors I know. They had parallel query even before version 8i. PostgreSQL only had it recently. However, Oracle database sales team has been trying to piss off their customers for awhile now.I know that several banks in Asean region are using PostgreSQL/MySQL for their new applications Also, many companies are banning new Oracle database implementations implicitly unless there’s no alternatives. People didn’t have much choice for choosing database software back then. It is vastly different now.
It’s a lot harder to pick the right database for new applications now. It used to be just Oracle and Oracle only. These days, you got to think of the nature and volume of the data/application. You also need to decide if your application requires ACID. If you are using NoSQL, there’s graph, key-value, document and columnar. You don’t really want to use the wrong database for your application. Using the wrong datastore for your application will result into sub-optimal performance or ugly workarounds that you wouldn’t like. For example, you could use MongoDB for text search. However, a better database for heavy text search use case would be Elasticsearch. Just because MongoDB have text indexes and is able to perform text search doesn’t mean it’s the right datastore for it. Heck, you could even use MySQL as a pure key-value storage. Databases is all about polyglot persistence these days.
I believe that in years to come, Oracle database will become the mainframe of database software. Highly profitable yet legacy piece of software whereby companies only use it because they were too afraid to use something else. For all my friends that still works actively on Oracle databases, perhaps it’s time to learn a new skill or move on to something else.
2016 Year End Thoughts
I began this blog more than 4 years ago, on 8th September 2012. My life has undergone a massive change since then. 4 years ago, I started my first job working with Oracle databases on AIX/Solaris O/S. Today, I work with mainly MongoDB/PostgreSQL/MySQL on FreeBSD/Linux O/S. Within this short timespan, proprietary software market share has reduced massively. For every proprietary software, there is a open-source version out there. Albeit, it might not be as enterprise ready as the paid counterparts, but hey, it’s free! Also, database is a lot more cooler to work on now!
This year has been a life changing experience for me. I did a lot of freelance database consulting work, relocated to a different continent, also working with in a team with very different culture compared to mine. The tech industry in London is a lot more vibrant than Singapore. The average quality of engineers are a lot higher than back home. The best thing that happened to me professionally is that I realised that I wasn’t as good as I thought. I’m happy that I had this self realisation myself instead of having it shoved down my throat. (Think of China during the Qing dynasty!)
My professional motto is always to be the dumbest guy in the room. So far, I was never the dumbest guy. Fortunately, there was always this few guys that’s just a tad dumber that me. However, in this new team, I do feel like the dumbest guy in the room. It’s probably more of a fact than feeling. It is brutal and humbling and definitely not a pleasant experience. It doesn’t help emotionally when the working culture are forthright, meaning they are more than willing to engage in a point-blank technical conversation without any hesitation of being as blunt as they can be.
However, it does kindle this burning sensation within me to strive to be a better engineer. I do want to read as much as I can and catch up to their level. I will let my inner asian mindset shine here 🙂 2016 has been an amazing ride so far, I can’t wait for 2017 to happen. It’s going to be so much more exciting!
I wish everyone to have an awesome year in 2017!
What to do when you join a company as a new DBA
When we take up a new job and join a company as a new DBA, what should we do immediately? Unless your manager has specific task or expectations for you, this is what I recommend. Start documenting or looking through all the database you will be managing. This should include important database information like the following:
1. Database architecture diagram (Is this RAC, DataGuard or GoldenGate?)
2. Database parameters (AMM or ASMM, Archive logging, DB block size, DBID?)
3. Understand the application using the database (OLTP, DWH, Hybrid?)
4. Read and understand existing documentations available. (DR, Failover, Backup procedures)
The reason is simple, understand the environment you will be managing throughly. Every database environment is different. By documenting the environment, you will be able to understand it fully. It will help when you are troubleshooting issues.
How did I became an (Oracle) Database Administrator
Every now and then, I get questions asking me how did I get into the field of database administration. Getting a role as a DBA is tricky. If you look at the requirements for DBA, they usually require more than 3 years of experience. There are very few companies looking for fresh graduates for DBA role.
After completing my 2 years of mandatory National Service (NS), I began to look for an IT job. 2 years of NS reduced my IT knowledge to non-existence. I was out of touch with the IT industry. I couldn’t even tell you what version was .NET framework at that time. I attended an interview (System Engineer) with a web-hosting company in Singapore. It was a horrible experience.
Interviewer: So, why do you want to join the IT industry?
Me: Because I love technology and I think it’s rather interesting.
He started asking me technical questions about the position and I could barely answer his questions. At the end of the interview, he made this remark.
I’m going to be frank with you because I do not want to waste your time. I will not hire you. You have no talent in IT. Since you mentioned that you are interested in technology, why don’t you go and try to sell handphones. It will probably suit you better.
I was devastated. I felt so insulted and shameful of myself. I was never insulted like this before. I went home and cried my lungs out. I didn’t had to mood to attend any interviews after that. I was seriously contemplating about what to do with my career. Then, my brother introduced me a 2 month free Oracle database course (MDeC KDI programme) in Malaysia, KL.
I worked my ass off for 2 months straight and got my first Oracle certification (OCA 11G DBA). When I got back to Singapore, I applied for many Oracle DBA positions and finally I got an interview from Seagate. It took me about 3 months of job hunting. The interview went like this.
He asked me 2 technical questions.
1. Purpose of Redo log
2. Purpose of Undo log
Interview: Why do you want to join us?
Me: Because I want to learn. The job scope is exactly what I am looking for. The salary does not matter to me, you can pay me $800 and I will still want the job.
And yes, I got hired in the end. 🙂
Today, I am still very grateful to my hiring manager at that time (I still drink coffee with him once in a while). He told me, the only reason why he hired me was because of my attitude, he knew that I was eager to prove myself and that was why I will be willing to put in everything I got to make it work. My advice to all fellow IT professionals, keep that learning attitude and your chance will come. Don’t give up and don’t get discouraged by the nay-sayers.
We have a saying in Chinese.
Chance is for those who are prepared.
Today, I found out that I made an extremely silly mistake.
This was a very simple project. A 2-Node Oracle RAC setup connected to a IBM v7000 storage. The deployment was smooth. I configured the ASM disk using udev and device-mapper. However, after the system was handover to the customer, they begin encountering reboots and the following error.
qla2xxx [0000:1b:00.0]-801c:1: Abort command issued nexus=1:1:0 -- 1 2002.
These were the steps I took to resolve the issue.
- Logging of service ticket to IBM, Red Hat and QLogic.
- Flashing of QLogic HBA firmware
- Patching of QLogic HBA driver
However, the issue was still not resolved. The root cause was infact that the udev was configured incorrectly! The SCSI timeout was supposed to be set to 120s according to IBM documentation. This was a mistake that made me angry with myself. These were the few reasons for my mistake
- Check and verify all configurations again. (This was the part I did not perform. I was VERY confident that I configured it correctly)
- Do not assume and bet on the root cause so quickly. (I was too quick to assume to root cause to be the firmware/driver issues)
- Do not be swayed by the customer no matter how experienced they are. Have your own understanding and judgement!
Where to get knowledge on databases (Oracle/MSSQL/MongoDB)?
Being in a IT profession is one of the most challenging career. There are many areas of IT you can focus on. For example, security, networking, systems, application and many more! The worst thing is that all these demands for skill sets changes over time. If you bet your luck on a single technology, you WILL be out of the job some day in the future. (hint: lotus notes)
However, that is also the most interesting aspects of being a IT professional. You learn new things and adapt/evolve to the ever changing landscape of the IT industry. I have less than 3 years of experience in IT and have yet to experience the downfall of any major technology. However, I firmly believe in the following;
- You need to embrace changes. If you stop changing, you will get changed!
- Be curious to all things
- Be open-minded to new technologies
I would also like to recommend some goodies for my readers. The following lists are where I get my usual “self-improvement” materials from. Do note that they are mostly database related. Its a database blog after all!
Globalisation is happening around the globe where IT team can be in anywhere supporting the business. We are competing with IT professionals globally. This does not just happen in IT industry alone. Being knowledgable in more than one database technology will definitely helps in the long run.
Ever since I started my career in Oracle database, I have always been wondering why is there no Oracle user community in Singapore. There are many different Oracle user group in the world. IOUG, UKOUG and even PKOUG! Singapore is one of the biggest financial and IT hub in the APAC region, there is absolutely no reason not to have our own SGOUG!
I really think that it would be nice if we can have our own Oracle community in Singapore where we can share ideas and learn from each other. I have started this little group in Facebook to see if we have enough people in Singapore to start our own Oracle user group.
If you are located in Singapore and is working with Oracle technology or want to learn more about Oracle technology, please do join our group. If we have enough people, I will try to set a regular meetup every month!
See you then!
Oracle Weblogic Server Extreme Performance and Innovation Architecture Workshop on 4th October 2013
As The Extreme Architecture/Development Performance Workshop will share with practitioners use cases and best practices for designing high performance architectures using WebLogic Server. The workshop will also share with practitioners with deep dive advanced performance tuning practices at the WebLogic Server level, Java Virtual Machine and operation level.
We will describe typical use cases and patterns in implementing a WebLogic Server based architectures for extreme performance through a number of case studies with everything put in its proper context. Performance issues will be discussed in detail covering symptoms and business impacts to problem identification and resolution. We will then explore lessons learned from solving performance problems through design changes and against other products in the marketplace.
Following on from last year’s WebLogic Server tuning deep dive theme, we will discuss the tuning aspect from a scalability and topology requirement. We will take you through the capability of WebLogic Server distributing a service over clustering and the performance consideration of how dynamic clustering work load management can provide quality of service when under load. Details of cluster propagation, routing, and selection for various communication protocols will also be covered. We will examine the performance improvement from high availability features of WebLogic Server. Is never easy when migrating or upgrading from an older version of WebLogic or another application server. We will share with you the tips and tricks of the development effort necessary to keep your application running at extreme performance.
The Java Virtual Machine underpins WebLogic Server and serves a broad spectrum of Oracle’s Fusion Middleware products. Over the years, JVM tuning has been well understood. However, little is known with the G1 garbage collector that was introduced in JDK 7. We will discuss how the G1 collector function internally, the key command line switches for using G1 and options for logging its operation. You will also learn the difference between G1 and other collectors from a performance aspect as well as how to exploit G1 for optimised performance. Additional, we’ll demonstrate the monitoring tools and framework used to monitor a running JVM.
At an operation/deployment level, you will learn how to make WebLogic Server perform to its potential by collecting performance test data and analysing them for common cause of performance issues through our Enterprise Manager. We will present the best practices for designing and using Enterprise Manager as a proactive monitoring tool for your WebLogic architecture with the view of maximising performance and with minimal overhead. A sample scenario will be used to demonstrate an end-to-end transaction monitoring from an operation level through to problem diagnose.
Where applicable we will use demonstrations that showcase the capabilities and concepts illustrated in the various sessions. By the end of the workshop you will be able to design, tune, develop and manage WebLogic Server at extreme performance.
Register over here!
Oracle Database Workshop on 23rd October 2013
IT innovation and efficiencies allow organizations to maximize the use of their existing assets as they move towards more agile and cost-effective technology environments. IT organizations are striving to reduce total cost of ownership (TCO), automate IT management, improve their ability to react to changing business demands, and at the same time, working to meet new compliance mandates.
Discovering and implementing new IT efficiencies offers avenues for optimizing IT operations across the organization, while at the same time, improving the quality of IT services with reduced cost, increased effectiveness and greater security.
Register over here!
Hope to see you over there!