Network Debugging Utility – tcpdump

tcpdump tool helps to see all or certain packets going over the Ethernet to debug network problems. It needs to be run as root under Linux in order to be able to sniff network packets.
The range of packets captured can be specified by the using a combination of logical operators and parameters such as source and destination IP addresses, protocol types and TCP/UDP port numbers.

tcpdump output has the following format,

For UDP packets

20:12:40.623187 csqlcache.com.3050> 192.168.1.51.34353: udp 234

where

Timestamp  20:12:40.623187
Source address csqlcache.com
Source port 3050
Destination address  192.168.1.51
Destination port 34353
Protocol udp
Packet Size  234

For TCP packets

20:12:40.623187 IP(…) csqlcache.com.3051 > 192.168.1.51.34353: P1:53(52)) ack 168 win 64315 

where

Timestamp  16:23:01.079553
Protocol IP
Source address csqlcache.com
Source port 3051
Destination address 192.168.1.51
Destination port 34353
Sequence number 1
Number of user data bytes in datagram (52)

To capture all traffic with the tcp or udp source or destination number 22 (ssh port), run the following command

$tcpdump port 22
It displays the following output

0:12:40.623084 IP (tos 0x10, ttl 64, id 34755, offset 0, flags [DF], proto TCP (6), length 92) 192.168.1.113.ssh > 192.168.1.103.appserv-https: P 3533382018:3533382070(52) ack 1574531805 win 9648
20:12:40.699411 IP (tos 0x10, ttl 64, id 34756, offset 0, flags [DF], proto TCP (6), length 156) 192.168.1.113.ssh > 192.168.1.103.appserv-https: P 52:168(116) ack 1 win 9648
20:12:40.623187 IP (tos 0x0, ttl 128, id 5012, offset 0, flags [DF], proto TCP (6), length 40) 192.168.1.103.appserv-https > 192.168.1.113.ssh: ., cksum 0x57ec (correct), ack 52 win 64431
20:12:40.777047 IP (tos 0x0, ttl 128, id 5013, offset 0, flags [DF], proto TCP (6), length 40) 192.168.1.103.appserv-https > 192.168.1.113.ssh: ., cksum 0x57ec (correct), ack 168 win 64315
20:12:41.506720 IP (tos 0x0, ttl 128, id 5014, offset 0, flags [DF], proto TCP (6), length 92) 192.168.1.103.appserv-https > 192.168.1.113.ssh: P 1:53(52) ack 168 win 64315
From this you can derive meaningful information for doing network analysis of your programs. This tool will be very helpful for debugging socket programs.

There are many options provided for this tool, for example

To capture all TCP traffic with source address csqlcache

$tcpdump tcp src host csqlcache
For complete set of options, please visit http://www.tcpdump.org/

Techniques to reduce memory footprint

Avoid using recursive calls

Recursive calls grow the stack memory of the process and when it is deeply nested, it may lead to stack overflow.

Prefer lazy loading than initial loading.

For example, if we need to count the number of words in a file, it is better to read the data line rather than reading the whole file into memory and then count the words.Readingline by line into the same buffer and processing each line can do this.

Prefer Shared libraries than static libraries

When shared library is used, the physical pages associated with the shared libraries are shared for all the process using that library. But in case of static libraries, each executable has its own copy leading to huge memory utilization for all the process using that library.

Modularizing and on-demand loading

Instead of linking your executable with your big-shared library, which contains all the functionality, we can modularize and create multiple shared libraries, which shall be loaded on demand based on the configuration. One issue with this, the performance will degrade as the program will do dlsym() to locate the symbol in the library. This we shall overcome by populating the function pointer list only once by providing the initialization function which does dlsym for all the necessary symbol and store it in lookup table or array of function pointers.

Use bit mask to store flags rather than bool or char data type

When multiple options need to be specified, it is usually defined as set of integer or character values that takes lot of space

struct Options{

bool isFlag1;

bool isFlag2;

bool isFlag3;

bool isFlag4;

bool isFlag5;

};

Instead, it can be stored in bit mask as follows

struct Options {

char opt;

};

#define FLAG1   1

#define FLAG2   2

#define FLAG1   4

#define FLAG1   8

#define FLAG1   16

Using the macro values defined above you can set the options by OR’ing values to it.

Network Debugging Utility – netcat

nc or netcat utility is command line tool for debugging server or client network application. It can open TCP connections, send UDP packets, listen on TCP or UDP port,etc.

 Syntax

nc [options] [host] [port]

It is very easy to build basic client server model using this tool.  To transfer file from one host to another,

On the host where you have the file (file.txt) to be transferred, run

$nc localhost 2000 <file.txt

On host where you want to get the file, run

$nc –l 2000 >file.txt

-l option instructs to listen on the specified port.

It is sometimes useful to talk to servers through command prompt.  It helps in troubleshooting, to verify what data a server is sending in response to commands issued by the client.  For example, to retrieve the home page of a web site:

           $ echo -n “GET / HTTP/1.0\r\n\r\n” | nc http://www.yahoo.com 80

It will display the response sent by the web server. If we know the format of the requests required by the server, we can interactively send and get response using this tool.

It can also be used to do port scanning on any hosts. Port scanning is process of connecting to ports and reporting whether the service or server process is running or not.

$nc -z localhost 1-5000

Output

Connection to localhost 22 port [tcp/ssh] succeeded!

Connection to localhost 25 port [tcp/smtp] succeeded!

Connection to localhost 111 port [tcp/sunrpc] succeeded!

Connection to localhost 631 port [tcp/ipp] succeeded!

Connection to localhost 1521 port [tcp/ncube-lm] succeeded!

Connection to localhost 3306 port [tcp/mysql] succeeded!

Another option that may come in handy is the -c option, which tells netcat to execute a command with /bin/sh after it connects — sending the output to the other side of the connection. This can be used on either side of the connection.

To send data from a command to a remote host, you could use

$netcat -c ‘/bin/command’ hostname port

When netcat connects to the service on the remote host, it will attempt to send the output of /bin/command

If you use netcat -l -p 1234 -c ‘/bin/command’, it will send the output of /bin/command to the first client that connects to port 1234, and then close the connection

How to detect memory leaks and memory corruption

Valgrind will save you hours of debugging time spent on analysing memory corruption. With valgrind tools you can detect many memory management and threading bugs. This gives you a feel that your programs are free of many common bugs. You can find and eliminate bugs before they become a problem with valgrind. Valgrind is a debugging tool for large, complex programs.

When to use valgrind?

  • For small programs with short run-times, when developing you can always run the program under a valgrind tool (usually Memcheck), knowing that memory bugs will be found immediately.
  • In automatic testing. By using Valgrind tools in your automatic unit, integration, system you can be confident no code will be unchecked.
  • After big changes. To ensure new bugs haven’t been introduced in the new code.
  • When a bug occurs. Get instant feedback about t the bug , where it occurred, and why.

Examples:

Ex. //test.c

int main()
{
char *p=malloc(100);
}

$gcc -c val.c

$gcc -osingo val.o

$ valgrind –tool=memcheck ./singo

output:

==16405== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 12 from 1)

==16405== malloc/free: in use at exit: 100 bytes in 1 blocks.

==16405== malloc/free: 1 allocs, 0 frees, 100 bytes allocated.

==16405== For counts of detected errors, rerun with: -v

==16405== searching for pointers to 1 not-freed blocks.

If you have a memory leak, then the number of allocs and the number of frees will differ . If the number of allocs differs from the number of frees, you can rerun your program again with the leak-check option. This will show you all of the calls to malloc/new/etc that don’t have a matching free.
Compile your program with -g to include debugging information so that Memcheck’s error messages include exact line numbers.

$gcc -g -o val1 val1.c

$valgrind –tool=memcheck –leak-check=yes .val1

output:

==19466== malloc/free: in use at exit: 100 bytes in 1 blocks.

==19466== malloc/free: 1 allocs, 0 frees, 100 bytes allocated.

==19466== For counts of detected errors, rerun with: -v

==19466== searching for pointers to 1 not-freed blocks.

==19466== checked 47,596 bytes.

==19466==

==19466== LEAK SUMMARY:

==19466== definitely lost: 0 bytes in 0 blocks.

==19466== possibly lost: 0 bytes in 0 blocks.

==19466== still reachable: 100 bytes in 1 blocks.

==19466== suppressed: 0 bytes in 0 blocks.

==19466== Reachable blocks (those to which a pointer was found) are not shown.

==19466== To see them, rerun with: –leak-check=full –show-reachable=yes

Reading/writing off the end of malloc’d blocks

Valgrind can also find the use of invalid heap memory using the memcheck tool. For instance, if you allocate an array with malloc or new and then try to access a location past the end of the array:

Ex. //val.c

main()

{

char *p=malloc(100);

p[120]=’b’;

}

output:

==19489== Invalid write of size 1

==19489== at 0x80483EA: main (val1.c:6)

==19489== Address 0x40260A0 is not stack’d, malloc’d or (recently) free’d

Reading/writing memory after it has been free’d :

main()

{

char *p=malloc(100);

free(p);

p[23]=’b’;

}

output:

==2808== Invalid write of size 1

==2808== at 0x80483EA: main (val1.c:6)

==2808== Address 0x40270a0 is not stack’d, malloc’d or (recently) free’d

Use of uninitialized memory:

main()

{

int x;

x++;

printf(“%d”,x);

}

output:

==3276== Use of uninitialised value of size 4

==3276== at 0x3A27DB: _itoa_word (in /lib/libc-2.7.so)

==3276== by 0x3A673F: vfprintf (in /lib/libc-2.7.so)

==3276== by 0x3AC9B2: printf (in /lib/libc-2.7.so)

==3276== by 0x80483EB: main (in /home/lakshya/rima/a.out)

Reading/writing inappropriate areas on the stack

main()

{

int s;

int *p=&s;

p[-2]=7;

s=p[-20];

}

output:

==3313== Invalid read of size 4

==3313== at 0x80483BD: main (in /home/lakshya/rima/a.out)

==3313== Address 0xbeb5e5bc is just below the stack ptr. To suppress, use: –workaround-gcc296-bugs=yes

Overlapping src and dst pointers in memcpy() and related functions:

 

ex:

main()

{

char buf[1000];

char *ptr1=&buf[0];

char *ptr2=&buf[400];

memcpy(ptr1,ptr2,500); 

}

output:

==3362== Source and destination overlap in memcpy(0xBE9ED224, 0xBE9ED3B4, 500)

==3362== at 0x4007CB2: memcpy (mc_replace_strmem.c:402)

==3362== by 0x804840B: main (in /home/lakshya/rima/a.out)

Passing unaddressable bytes to a system call:

Ex:

main()

{

char *ptr=(char *)malloc(100);

read(0,ptr,500);

write(1,ptr,500);

}

output:

==3382== Syscall param read(buf) points to unaddressable byte(s)

==3382== at 0x42B343: __read_nocancel (in /lib/libc-2.7.so)

==3382== by 0x37D38F: (below main) (in /lib/libc-2.7.so)

==3382== Address 0x402708c is 0 bytes after a block of size 100 alloc’d

==3382== at 0x4005AC8: malloc (vg_replace_malloc.c:207)

==3382== by 0x8048440: main (in /home/lakshya/rima/a.out)

 

==3382== Syscall param write(buf) points to uninitialised byte(s)

==3382== at 0x42B3C3: __write_nocancel (in /lib/libc-2.7.so)

==3382== by 0x37D38F: (below main) (in /lib/libc-2.7.so)

==3382== Address 0x4027033 is 11 bytes inside a block of size 100 alloc’d

==3382== at 0x4005AC8: malloc (vg_replace_malloc.c:207)

==3382== by 0x8048440: main (in /home/lakshya/rima/a.out)

 

Mismatched use of malloc/new/new [] vs free/delete/delete []

#include “stdlib.h”

main()

{

int *ptr=new int;

free(ptr);

}

output:

==3426== Mismatched free() / delete / delete []

==3426== at 0x400566C: free (vg_replace_malloc.c:323)

==3426== by 0x80484EE: main (in /home/lakshya/rima/a.out)

==3426== Address 0x4028028 is 0 bytes inside a block of size 4 alloc’d