16. Troubleshooting the problems

This post describes how to you troubleshoot the any problems.

As mention on the post 4. UML that simple diagrams will save you lot of time for the explanation. So I am going to draw diagram here to show how we can troubleshoot the problems if website or application is not working and the same troubleshooting funda can applicable for other problems too.

Diagram:

As of now, I can think of that the following other ways to help to troubleshoot the problem.

Notes:

1. Use Fiddler to inspect your HTTP traffic.
2. Check Event Viewer and exception logging database if you have for that website or application.
3. If your website hosted on Web Farm(same website hosted on multiple web servers), then very hard to find the request coming from which web server, So in order to trouble shoot the problems easily I would suggest to define the customer http headers on IIS using “HTTP Response Headers”  for each web servers. For example, please refer the below picture and you will understand that.

4. If your website or application build up with many layers (say UI, BAL, Web Service, Data, caching …etc.) then I would suggest to write the simple script to make sure everything is working on individual layers including connectivity and calling simple methods…etc. Because this will save us lot of time. For example, if there is a problem on DB then if we execute the script and script will tell us straightaway where is the problem instead of looking from web servers and narrow down the issue to other layers.

5.  Setup instant notifications via email or SMS for your production environments if any critical problems are happened or any health check issues.

6. Prepare the support manual documentation and use it for when the problems are happened and this document also very useful for future support/developers.

7. To work our applications on all of the modern browsers, sometimes, we need to write separate working block for the some of the functionalities for specific browsers. For example, our testers say it’s not working on Chrome or IE…etc. then we need fine the alternative working solution of that.

8. Please use networking commands to troubleshoot the problems for network related issues.

I hope you enjoy reading this post. Please contact me if you have any questions.

Architect Graph

Architect Topics Graph

Please click the below links which do you wish to know more details to become an Architect.

1. Coding standards & practices

2. OOAD

3. Design patterns

4. UML

5. Security

6. R&D for up to date technologies

7. Database design

8. Feeding the people technically

9. Testing framework

10. Documentation skills

11. Communication skill

12. Doing Certifications

13. SOA & Cloud Fundamentals

14. Building SOA & Cloud based services

15. Networking skills up to Architect level

16. Troubleshooting the problems

17. Lesson learned from your experience

Contact

If you have any questions regarding about design/architect the software systems or any other technical help, please contact Jayachandran(Jay) at Jayachandran.NET@hotmail.com. He will  reply your queries as soon as he can.

Architect Help

If you have any questions regarding about design/architect the software systems or any other technical help, please contact Jayachandran(Jay) at Jayachandran.NET@hotmail.com. He will  reply your queries as soon as he can.

Technical Bricks For Building Websites

What are the technical key considerations needed for building websites?

In overall, we need the following technical bricks to build the websites

1. Software Design(including choosing the software technologies which comply with modern technologies)
2. Development Environment Setup (including source version control, coding standards…etc.)
3. Authentication and Authorization(for frontend and backend layers depends upon the type of websites)
4. Security (including for preventing XSS, CSRF…etc.)
5. Exception Handling and Logging (including custom error messages, page not found error page…etc.)
6. Database Design (including normalization, store procedures, triggers, views…etc.)
7. Caching (for improving the performance of the web site)
8. Testing (including unit test, performance, Data Driven Testing (DDT)...etc.)
9. Hosting Environment Plan
10. Documentation (including deployment, support…etc. documentations)

Next, do we need necessarily all the above bricks for building websites?  

Yes. But no. It's depends upon the type of websites.

9. Testing framework

Here, Testing Framework means various types of testing methodologies enforce for whatever the code written & each and every piece of code should be testable.

Testing is partially NFR (Non- Functional Requirements).

What is the NFR?

NFR is also called “ilities”. Because of the use of suffix “ility” with many non-functional requirements. For example, some of the NFR are Documentation, Availability, Maintainability, Interoperability, Recoverability, Supportability and testability…etc.

Why NFR is so important and how they are identified?

1. In our SDLC, we are always focusing and working on functional requirements (FR). In most of the projects, Project Owners mainly focus on FR (Functional Requirements) only. It’s important to give same level of preference for NFR after development finish. NFR are identifies during whole project life cycle and it’s very important to spend some time to put on the place. NFR helps lot of our development time and money for future projects.
2. Identifies  and Prioritize NFR  items like “Must”, ‘High Want” “Want” during project life cycle and implement NFR items depends upon the time availability.
3. NFR also includes Lesson learned such as what is necessary to document and put the methodologies needed for the future projects.
4. NFR items are not visible to business owners and they are not interested on this. However, project development team responsible to notify to business owner about the benefits of NFR.

Software testing types:

Please refer the below diagram.

Do we need to focus on all the above testings?

Yes but no. It is depends upon your project & project type.

I am not going to describes all the testing here. I do describe the necessary testing performed by development team.

Most of the testings are performed by test team. However, Development team should have testing frameworks for testing below tests.

Unit Testing: 

Developers are writing and running unit tests while they write their code to produce the high quality code. As we know, it takes times to write unit test. However we should be writing unit testing for each and every method we write.

There is a unit testing project comes with recent Microsoft Visual Studio release got many features. Please watch the below video for more information.

http://channel9.msdn.com/Events/TechEd/Europe/2012/DEV214

We need to use different kinds of testing frameworks depend upon our projects. For example, Rhino Mock-up framework is good for if your project need to authenticate with user name and password to test. 

Please refer the below URL

http://en.wikipedia.org/wiki/List_of_unit_testing_frameworks

for more information about testing frameworks.

And also refer the below URL’s for more information about unit testing.

http://msdn.microsoft.com/en-us/library/dd264975.aspx

http://blogs.msdn.com/b/visualstudioalm/archive/2012/11/09/how-to-manage-unit-tests-in-visual-studio-2012-update-1-part-1-using-traits-in-the-unit-test-explorer.aspx

http://en.wikipedia.org/wiki/Visual_Studio_Unit_Testing_Framework

Integration Testing:

Each individual’s software modules are combined and tested a group. It does occur after unit testing. 

Please refer the below URL’s for more about integration testing.

http://msdn.microsoft.com/en-us/magazine/jj618302.aspx

http://msdn.microsoft.com/en-us/library/ms182470.aspx

http://msdn.microsoft.com/en-us/library/ms182409.aspx

http://blog.stevensanderson.com/2009/06/11/integration-testing-your-aspnet-mvc-application/

Performance Testing:

It’s very important & necessary test to perform for each and every application. There are different types of performance testing measures the application performance.

http://blogs.msdn.com/b/visualstudioalm/archive/2012/06/04/getting-started-with-load-testing-in-visual-studio-2012.aspx

Please note that all the Microsoft Visual Studio editions does not support for performance testing. Please refer the below URL for more information and buy relevant product which is suitable for your project to perform the performance test.

http://www.microsoft.com/visualstudio/eng/products/compare

It’s good to have performance testing performed for each layer. Let’s say, if your project has UI, BAL, web Service, Data layers then please try to develop the performance test for individual layer to test the performance test.

Please refer the below URL’s

http://msdn.microsoft.com/en-us/magazine/jj190803.aspx

http://msdn.microsoft.com/en-us/library/vstudio/dd293540.aspx

http://msdn.microsoft.com/en-us/library/ms404656(v=vs.80).aspx

Performance test objectives include the following:

Response time or latency

Throughput

Resource utilization

Please refer the below URL’s and you will get to know more about the objectives and measurements for the performance testing.

http://msdn.microsoft.com/en-us/library/bb924356.aspx

http://msdn.microsoft.com/en-us/library/ff647791.aspx

http://msdn.microsoft.com/en-us/library/ff647215.aspx

http://en.wikipedia.org/wiki/Software_performance_testing

Please refer the http://msdn.microsoft.com/en-us/library/bb924357.aspx to know more about the types of performance testing.

Performance vs. Scalability:

Performance refers to the capability of a system to provide a certain response time.

Scalability refers to the characteristic of a system to increase performance by adding additional resources such as processor, memory, and web server in web farms.

Please visit the following URL’s

http://www.opensourcetesting.org/performance.php

https://developers.google.com/speed/pagespeed/ to expert in performance testing.

Compatibility Testing:

Now days, each and every application which we develop should be compatible for mobile devices as most of the traffic coming from mobile devices. To verify our application how works on mobile devices, we can use browsers extensions (such as Firefox user agent switcher) which are available on the market. However, some situations/scenarios these testing tools does not simulate 100% correctly. For testing on real mobile devices on remotely please use

http://www.keynotedeviceanywhere.com

And other useful links

http://www.1stwebdesigner.com/design/tools-browser-compatibility-check/

http://browsershots.org/

http://www.browserstack.com/

Finally, I would suggest that to prepare testing check list for your team and use if for your project development.

I hope you enjoy reading this post.

7. Database design

In this post, I am going to talk about how database design is very important for any applications.

Database is the backbone for any application.  I am not going to write much theory here and do explain what are the necessary things to remember when we do the database design.

This below diagram describes the overall data modelling steps involved when we do the database design.

Please keep in mind the following database concepts and corresponding usage when database design or working on data base.

Concepts and description:

RDBMS (Relational Database Management System) :

Contains the tables and enforces relationships between the tables.

RDBMS Table Properties:  

1. Values are atomic.
2. Each Column must have a unique name
3. The sequence of rows and columns are insignificant.
4. Each column must specify same type of values

Normalization & different normalization forms:

The process of organizing data to minimize the redundancy is called normalization.

1^st Normal Form: Eliminate repeating groups. Define set of attributes for each table and set the primary key for the each table.

2^nd Normal Form: Eliminate redundant data.

3^rd Normal Form: Eliminate columns not dependent on the key.

4^th Norma Form:  Isolate independent multiple relationships.

Boyce- Codd Normal Form: If there are non-trivial dependencies between candidate key attributes, than separate them out into distinct tables.

5^th Normal Form: isolate semantically related multiple relationships.

Note:

1. First 3 normal forms should be necessarily implemented.
2. Too much normalization is not good for applications.
3. There are other normalizations namely Optimal and Domain-Key normal forms

De-normalization:

De-normalization is the process of attempting to optimizing performance of the database by adding redundant data.

Primary and Unique key:

Primary key is the unique identifier of the each row. Primary key prevents duplicate values. Primary key does not allow NULL values.

Unique + Not NULL = Primary Key

Unique key enforces the uniqueness of the values in the set of columns. Unique key allow on NULL value.

Candidate key:

The key which is become primary key is called candidate key.

Foreign key:

Foreign key enforces the referential integrity.  Foreign key in one table points to primary key or unique on another table.

ACID Database characteristics:

ACID: Atomicity, Consistency, Isolation, Durability. For more information, please refer http://en.wikipedia.org/wiki/ACID

Store Procedure and advantages:

Store procedure is the group of SQL statements that have been previously created and stored in the SQL Server. This is pre-compiled.

Advantages:

1. Reduce the network traffic.
2. More secure and prevents the SQL injection.
3. Speed and Modularization.

Trigger:

Trigger is the SQL procedure or SQL code which fire the action (When INSERT OR DELTE OR UPDATE on database). For more information, please refer http://en.wikipedia.org/wiki/Database_trigger

View:

View is the virtual tables and does not occupy any space. Views can be used for securing the columns fields. Please refer http://en.wikipedia.org/wiki/View_(database) for more information.

Index:

Index is used for fast data retrieval. Please refer http://en.wikipedia.org/wiki/Database_index for more information.

Cursor:

Cursor is a database object used by applications to manipulate data in a set on a row-by-row basis. Please refer the http://en.wikipedia.org/wiki/Cursor_(databases) for more information.

SQL Types:

Refer below diagram.

What is the difference between UNION and UNION ALL?

UNION:

The UNION command is used to select related information from two tables, much like the JOIN command. However, when using the UNION command all selected columns need to be of the same data type. With UNION, only distinct values are selected.

UNION ALL:

The UNION ALL command is equal to the UNION command, except that UNION ALL selects all values.

The difference between Union and Union all is that Union all will not eliminate duplicate rows, instead it just pulls all rows from all tables fitting your query specifics and combines them into a table.

What is difference between DELETE & TRUNCATE commands?

Delete command removes the rows from a table based on the condition that we provide with a WHERE clause. Truncate will actually remove all the rows from a table and there will be no data in the table after we run the truncate command.

TRUNCATE:

TRUNCATE is faster and uses fewer system and transaction log resources than DELETE.

TRUNCATE removes the data by deallocating the data pages used to store the table’s data, and only the page deallocations are recorded in the transaction log.

TRUNCATE removes all rows from a table, but the table structure, its columns, constraints, indexes and so on, remains. The counter used by an identity for new rows is reset to the seed for the column.

You cannot use TRUNCATE TABLE on a table referenced by a FOREIGN KEY constraint. Because TRUNCATE TABLE is not logged, it cannot activate a trigger.

TRUNCATE cannot be rolled back.

TRUNCATE is DDL Command.

TRUNCATE Resets identity of the table

DELETE:

DELETE removes rows one at a time and records an entry in the transaction log for each deleted row.

If you want to retain the identity counter, use DELETE instead. If you want to remove table definition and its data, use the DROP TABLE statement.

DELETE Can be used with or without a WHERE clause

DELETE Activates Triggers.

DELETE can be rolled back.

DELETE is DML Command.

DELETE does not reset identity of the table.

What is difference between a primary key and a unique key?

Both primary key and unique key enforces uniqueness of the column on which they are defined. But by default primary key creates a clustered index on the column, where are unique creates a non-clustered index by default. Another major difference is that, primary key doesn’t allow NULLs, but unique key allows one NULL only.

A clustered index is a special type of index that reorders the way records in the table are physically stored. Therefore table can have only one clustered index. The leaf nodes of a clustered index contain the data pages.

A non-clustered index is a special type of index in which the logical order of the index does not match the physical stored order of the rows on disk. The leaf node of a non-clustered index does not consist of the data pages. Instead, the leaf nodes contain index rows.

SQL Joins:

Please refer the below diagram

Please refer http://blog.sqlauthority.com/2009/04/13/sql-server-introduction-to-joins-basic-of-joins/ for more information about joins.

Linked Server:

Linked Server option enables to execute the commands against the OLE DB data sources on remote servers.

Locking:

Locking is a method used to protect records that will be accessed by multiple users and its prevents the concurrency errors.

There are two types of locking 1. Optimistic 2. Pessimistic

In pessimistic locking, when user wants to update data it locks the record and till then no one can update data. Other user’s can only view the data when there is pessimistic locking.

In optimistic locking multiple users can open the same record for updating, thus increase maximum concurrency. Record is only locked when updating the record. This is the most preferred way of locking practically. Now a days in browser based application it is very common and having pessimistic locking is not a practical solution.

Please refer

http://msdn.microsoft.com/en-us/library/aa213039(v=sql.80).aspx

http://www.dotnetfunda.com/articles/article1030-6-ways-of-doing-locking-in-net-pessimistic-and-optimistic.aspx

http://msdn.microsoft.com/en-us/library/777e5ebh.aspx

http://blog.sqlauthority.com/

http://blog.extremexperts.com

for more about locking, how you prevent concurrency problems and other useful information.

What is the difference between having and where clause?

1. Where clause cannot be used with aggregates, but the having clause can.
2. Having specifies a search condition for a group or an aggregate function used in SELECT statement.

Filtered index:

It uses a filter predicate to index a portion of rows in the table. Please refer http://technet.microsoft.com/en-us/library/cc280372.aspx for more information about filtered index.

What is the difference between varchar & nvarchar?

1. Nvarchar can store Unicode characters but varchar does not store the Unicode characters.
2. Varchar is stored as regular 8-bit data. But NVARCHAR strings are stored in the database as 16 bits
3. Use nvarchar when if wants to store the text in different languages.
4. Varchar is better performance than nvarchar

Can we do unit test for SQL server functions, store procedures, triggers…etc.?

Yes. Please refer http://msdn.microsoft.com/en-us/library/aa833169.aspx

http://blogs.msdn.com/b/atverma/archive/2010/07/28/how-to-unit-test-sql-server-2008-database-using-visual-studio-2010.aspx

for more information.

Please keep in mind the below notes when working on database.

Notes:

1. Use necessary data type needed for each column. For example, if you are not going to store the different language text for that column then you no need nvarchar. In this case, use varchar.
2. Use normalization
3. Write unit test for your store procedures, triggers, functions and evaluate the execution time for each and every query.
4. Use SQL Server performance tuning dashboard
5. Use http://www.opensourcetesting.org/performance.php for health check and performance issues.
6. Use SQL profiler for trouble shooting the performance issues.
7. Use SQL Server Best Practices Analyser tool for checking installation, configurations…etc.
8. Use Microsoft BI tools such as SSIS, SSRS, and SSAS if needed and use them appropriately.
9. Please prepare the checklist for your DB development and activities and use them whenever you deploy your DB package on production.
10. Please refer

http://msdn.microsoft.com/en-us/library/ff648235.aspx

http://msdn.microsoft.com/en-au/library/ms159689(v=sql.100).aspx

I hope you enjoy this post. I am thinking of writing another blog about Microsoft BI tools such as SSIS, SSRS, and SSAS if I get the time.