#3 Linkers, Loaders and Libraries

I'm recently reading the book "Linkers, Loaders and Libraries". It really opens my mind. The process of the generation of a program always confuses me a lot. To understand and put my learning into practice, this paper is to record my study on Chapter 3 of the book.

My programming environment is Fedora 20, 64bit.

1. gcc -c SimpleSection.c
2. objdump -h SimpleSection.o

SimpleSection.o:     file format elf64-x86-64

Sections:
Idx Name          Size      VMA               LMA               File off  Algn
  0 .text         00000054  0000000000000000  0000000000000000  00000040  2**2
                  CONTENTS, ALLOC, LOAD, RELOC, READONLY, CODE
  1 .data         00000008  0000000000000000  0000000000000000  00000094  2**2
                  CONTENTS, ALLOC, LOAD, DATA
  2 .bss          00000004  0000000000000000  0000000000000000  0000009c  2**2
                  ALLOC
  3 .rodata       00000004  0000000000000000  0000000000000000  0000009c  2**0
                  CONTENTS, ALLOC, LOAD, READONLY, DATA
  4 .comment      0000002d  0000000000000000  0000000000000000  000000a0  2**0
                  CONTENTS, READONLY
  5 .note.GNU-stack 00000000  0000000000000000  0000000000000000  000000cd  2**0
                  CONTENTS, READONLY
  6 .eh_frame     00000058  0000000000000000  0000000000000000  000000d0  2**3
                  CONTENTS, ALLOC, LOAD, RELOC, READONLY, DATA

3. size SimpleSection.o

   text   data    bss    dec    hex filename
    176      8      4    188     bc SimpleSection.o

4. objdump -x -s -d SimpleSection.o

SimpleSection.o:     file format elf64-x86-64
SimpleSection.o
architecture: i386:x86-64, flags 0x00000011:
HAS_RELOC, HAS_SYMS
start address 0x0000000000000000

Sections:
Idx Name          Size      VMA               LMA               File off  Algn
  0 .text         00000054  0000000000000000  0000000000000000  00000040  2**2
                  CONTENTS, ALLOC, LOAD, RELOC, READONLY, CODE
  1 .data         00000008  0000000000000000  0000000000000000  00000094  2**2
                  CONTENTS, ALLOC, LOAD, DATA
  2 .bss          00000004  0000000000000000  0000000000000000  0000009c  2**2
                  ALLOC
  3 .rodata       00000004  0000000000000000  0000000000000000  0000009c  2**0
                  CONTENTS, ALLOC, LOAD, READONLY, DATA
  4 .comment      0000002d  0000000000000000  0000000000000000  000000a0  2**0
                  CONTENTS, READONLY
  5 .note.GNU-stack 00000000  0000000000000000  0000000000000000  000000cd  2**0
                  CONTENTS, READONLY
  6 .eh_frame     00000058  0000000000000000  0000000000000000  000000d0  2**3
                  CONTENTS, ALLOC, LOAD, RELOC, READONLY, DATA
SYMBOL TABLE:
0000000000000000 l    df *ABS* 0000000000000000 SimpleSection.c
0000000000000000 l    d  .text 0000000000000000 .text
0000000000000000 l    d  .data 0000000000000000 .data
0000000000000000 l    d  .bss 0000000000000000 .bss
0000000000000000 l    d  .rodata 0000000000000000 .rodata
0000000000000004 l     O .data 0000000000000004 static_var.1728
0000000000000000 l     O .bss 0000000000000004 static_var2.1729
0000000000000000 l    d  .note.GNU-stack 0000000000000000 .note.GNU-stack
0000000000000000 l    d  .eh_frame 0000000000000000 .eh_frame
0000000000000000 l    d  .comment 0000000000000000 .comment
0000000000000000 g     O .data 0000000000000004 global_init_var
0000000000000004       O *COM* 0000000000000004 global_uninit_var
0000000000000000 g     F .text 0000000000000021 func1
0000000000000000         *UND* 0000000000000000 printf
0000000000000021 g     F .text 0000000000000033 main


Contents of section .text:
 0000 554889e5 4883ec10 897dfc8b 45fc89c6  UH..H....}..E...
 0010 bf000000 00b80000 0000e800 000000c9  ................
 0020 c3554889 e54883ec 10c745fc 01000000  .UH..H....E.....
 0030 8b150000 00008b05 00000000 01c28b45  ...............E
 0040 fc01c28b 45f801d0 89c7e800 0000008b  ....E...........
 0050 45fcc9c3                             E...          
Contents of section .data:
 0000 54000000 55000000                    T...U...      
Contents of section .rodata:
 0000 25640a00                             %d..          
Contents of section .comment:
 0000 00474343 3a202847 4e552920 342e382e  .GCC: (GNU) 4.8.
 0010 32203230 31333132 31322028 52656420  2 20131212 (Red
 0020 48617420 342e382e 322d3729 00        Hat 4.8.2-7).
Contents of section .eh_frame:
 0000 14000000 00000000 017a5200 01781001  .........zR..x..
 0010 1b0c0708 90010000 1c000000 1c000000  ................
 0020 00000000 21000000 00410e10 8602430d  ....!....A....C.
 0030 065c0c07 08000000 1c000000 3c000000  .\..........<...
 0040 00000000 33000000 00410e10 8602430d  ....3....A....C.
 0050 066e0c07 08000000                    .n......      

Disassembly of section .text:

0000000000000000 <func1>:
   0: 55                   push   %rbp
   1: 48 89 e5             mov    %rsp,%rbp
   4: 48 83 ec 10           sub    $0x10,%rsp
   8: 89 7d fc             mov    %edi,-0x4(%rbp)
   b: 8b 45 fc             mov    -0x4(%rbp),%eax
   e: 89 c6                 mov    %eax,%esi
  10: bf 00 00 00 00       mov    $0x0,%edi
11: R_X86_64_32 .rodata
  15: b8 00 00 00 00       mov    $0x0,%eax
  1a: e8 00 00 00 00       callq  1f <func1+0x1f>
1b: R_X86_64_PC32 printf-0x4
  1f: c9                   leaveq
  20: c3                   retq

0000000000000021 <main>:
  21: 55                   push   %rbp
  22: 48 89 e5             mov    %rsp,%rbp
  25: 48 83 ec 10           sub    $0x10,%rsp
  29: c7 45 fc 01 00 00 00 movl   $0x1,-0x4(%rbp)
  30: 8b 15 00 00 00 00     mov    0x0(%rip),%edx        # 36 <main+0x15>
32: R_X86_64_PC32 .data
  36: 8b 05 00 00 00 00     mov    0x0(%rip),%eax        # 3c <main+0x1b>
38: R_X86_64_PC32 .bss-0x4
  3c: 01 c2                 add    %eax,%edx
  3e: 8b 45 fc             mov    -0x4(%rbp),%eax
  41: 01 c2                 add    %eax,%edx
  43: 8b 45 f8             mov    -0x8(%rbp),%eax
  46: 01 d0                 add    %edx,%eax
  48: 89 c7                 mov    %eax,%edi
  4a: e8 00 00 00 00       callq  4f <main+0x2e>
4b: R_X86_64_PC32 func1-0x4
  4f: 8b 45 fc             mov    -0x4(%rbp),%eax
  52: c9                   leaveq
  53: c3                   retq

6. readelf -h SimpleSection.o

ELF Header:
  Magic:   7f 45 4c 46 02 01 01 00 00 00 00 00 00 00 00 00
  Class:                             ELF64
  Data:                              2's complement, little endian
  Version:                           1 (current)
  OS/ABI:                            UNIX - System V
  ABI Version:                       0
  Type:                              REL (Relocatable file)
  Machine:                           Advanced Micro Devices X86-64
  Version:                           0x1
  Entry point address:               0x0
  Start of program headers:          0 (bytes into file)
  Start of section headers:          400 (bytes into file)
  Flags:                             0x0
  Size of this header:               64 (bytes)
  Size of program headers:           0 (bytes)
  Number of program headers:         0
  Size of section headers:           64 (bytes)
  Number of section headers:         13
  Section header string table index: 10

7. readelf -s SimpleSection.o

Symbol table '.symtab' contains 16 entries:
   Num:    Value          Size Type    Bind   Vis      Ndx Name
     0: 0000000000000000     0 NOTYPE  LOCAL  DEFAULT  UND
     1: 0000000000000000     0 FILE    LOCAL  DEFAULT  ABS SimpleSection.c
     2: 0000000000000000     0 SECTION LOCAL  DEFAULT    1
     3: 0000000000000000     0 SECTION LOCAL  DEFAULT    3
     4: 0000000000000000     0 SECTION LOCAL  DEFAULT    4
     5: 0000000000000000     0 SECTION LOCAL  DEFAULT    5
     6: 0000000000000004     4 OBJECT  LOCAL  DEFAULT    3 static_var.1728
     7: 0000000000000000     4 OBJECT  LOCAL  DEFAULT    4 static_var2.1729
     8: 0000000000000000     0 SECTION LOCAL  DEFAULT    7
     9: 0000000000000000     0 SECTION LOCAL  DEFAULT    8
    10: 0000000000000000     0 SECTION LOCAL  DEFAULT    6
    11: 0000000000000000     4 OBJECT  GLOBAL DEFAULT    3 global_init_var
    12: 0000000000000004     4 OBJECT  GLOBAL DEFAULT  COM global_uninit_var
    13: 0000000000000000    33 FUNC    GLOBAL DEFAULT    1 func1
    14: 0000000000000000     0 NOTYPE  GLOBAL DEFAULT  UND printf
    15: 0000000000000021    51 FUNC    GLOBAL DEFAULT    1 main

8. nm SimpleSection.o

0000000000000000 T func1
0000000000000000 D global_init_var
0000000000000004 C global_uninit_var
0000000000000021 T main
                 U printf
0000000000000004 d static_var.1728
0000000000000000 b static_var2.1729

Installation Halide on Fedora

Before installing Halide, we need to install llvm and clang first. On fedora, it seems easy for us to install those two packages. Actually, you might find llvm-config is missing although the command “sudo yum install llvm” worked. So we'd better use the following commands from Halide's documents:

1. svn co https://llvm.org/svn/llvm-project/llvm/branches/release_32 llvm3.2

2. svn co https://llvm.org/svn/llvm-project/cfe/branches/release_32 llvm3.2/tools/clang

3. cd llvm3.2
4. ./configure --enable-optimized --enable-assertions --with-clang --enable-targets=x86,arm,nvptx

5. make -j8

6. edit the Makefile in Halide 

• export LLVM_CONFIG=<path to llvm>/Release+Asserts/bin/llvm-config
• export CLANG=<path to llvm>/Release+Asserts/bin/clang

7. cd Halide

8. make

9. make distrib

distrib

|-- halide.tgz

|-- include

|   |-- Halide.h

|   `-- HalideRuntime.h

`-- lib

    |-- libHalide.a

    `-- libHalide.so

You can find all you need from "distrib" directory!

My Fedora experience

I installed Fedora this January in order to experience linux. All these days I have met various problems and have solved all of them with google. Actually it seem meaningless to record those problem and related solutions again and again but it is of much meaning to record my experience. In other words, the following part becomes great to me because of me rather than solutions.

1. Disk

Smarctl (http://goo.gl/LsMMiJ) is famous for checking status of hard drive because I can always find it in many answers on the net. 
Disk in Fedora is also very good so for me it is to see the capacity of my disk.
Command line "df" can see all mounting hard drive.

2. RPM

It is common to delete or install rpm package. Usually I'd use the command "rpm -ivh ***.rpm". For deleting, first type command to search for installed package using "rpm -aq | grep ***" and then you can type "rpm -e ***". But command "yum remove ***" is highly recommended, especially for old kernels.

Qt Tips

Recently I have been developing with Qt. I don't want to waste the value of my time spent on searching for solutions to those problems, so recording all my quesitons will be much consoling.

1. QWebView loads html from resource
As you can search on the Net, many peole write codes like

webView->setUrl(QUrl::fromLocalFile(":/html/homepage.html"));

But it won't work. Using qDebug() to output its url, the only thing you can get is "about:blank". You should write like this:

webView->setUrl(QUrl("qrc:///html/homepage.html"));

P.S. To get your resource path, just open the resource editor in Qt creator, right-click on the target resource and choose "Copy Resource Path to clipboard".


to be continued...

How did I solve frequent crashing of libjpeg?

Recently I was using libjpeg to support my program but as most people met, it crashed frequently. At first time it crashed at jpeg_read_header() and I unexpectedly linked to another library solving this crash but met it again at jpeg_finish_decompress(). Although googling it hardly online, I got little help. Today, I used libjpeg-turbo (jpeg-static.lib) and suddenly found it working! What an exciting moment!

Platform: Windows XP professional
Compiler: Visual Studio 2010 C++
File: rw_dct.c, setup.py, rw_dct_test.py (I need to compile and get PYD file)

# setup.py
#!/usr/bin/env python2
import os
from distutils.core import setup, Extension
from distutils.sysconfig import get_python_lib

numpy_import = os.path.join(get_python_lib(), 'numpy', 'core', 'include')

mod_rw_dct = Extension('rw_dct',
                       sources=['rw_dct.c'],
                       libraries=['jpeg-static'],
                       include_dirs=[numpy_import])

setup(name='Get Pyd',
      version='0.1',
      description='Compiling Pyd file test',
      license='GNU GPLv3',
      ext_modules=[mod_rw_dct])

How to install StegoTool

I have spent five days on the installation work of StegoTool. To deal with problems, it was common for me to question on StackOverflow, although I seldom asked due to the thought that someone must already meet my problems. But this experience taught me that reading official documents is very important. Recording my installation will be much meaningful for me, during which I always got stuck.

Official Site of this StegoTool is here.

Python : python-2.7.5.msi
SciPy & NumPy : scipy-0.13.2-win32-superpack-python2.7.exe, numpy-1.8.0-win32-superpack-python2.7.exe
PIL : PIL-1.1.7.win32-py2.7.exe
Matplotlib : matplotlib-1.3.1.win32-py2.7.exe
PyQt : PyQt4-4.10.3-gpl-Py2.7-Qt4.8.5-x32.exe
Qt : qt-windows-opensource-5.1.1-mingw48_opengl-x86-offline.exe

Libjpeg : jpegsr8b.zip (after "./configure", generate exe from ckconfig.c to generate jconfig.h and then "make")
MSYS & MinGW : MSYS-1.0.11.exe, MinGw4.5.0

SIP : sip-4.15.3.zip (Don't use MSYS, all could be done in cmd. "python configure.py -p win32-g++", set "mingw/bin" in your environment variable and then "mingw32-make", "mingw32-make install")
SetupTools : setuptools-2.0.tar.gz
Dateutil : python-dateutil-2.2.tar.gz
Pyparsing : pyparsing-2.0.1.tar.gz

All above is what you will need to install the tool. Most of my time was spent on fixing endless problems after installing Enthought Canopy (academic version). I thought it would be convenient but the fact was the opposite. So this morning installing libraries seperately made it through. Here is how the tool goes:

If you are to run on windows like me, please be sure to do the following changes:

1. StegoTool.py

def load_crypto_schemes():

    schemedict = {}

    cryptodir = os.path.join(WORK_DIR, config.CRYPTOFOLDER)

    for scheme in os.listdir(cryptodir):

        if scheme != '__init__.py' and scheme.endswith('.py'):

            name = scheme[:-3]

            # please be sure that CRYPTOFOLDER is written in UNIX form

            mod_name = (config.CRYPTOFOLDER + name).replace('/', '.')

            try:

                mod = __import__(mod_name, fromlist=mod_name)

                schemedict[name] = getattr(mod, name)

            except ImportError:

                logger.warning('"%s is not a Python module!' % name)

            except AttributeError:

                logger.warning('"%s does not contain a corresponding class'

                               % name)

    return schemedict

2. qtwidgets.py

class ImageWidget(QtGui.QWidget):

    """ Widget to display an image """

    def __init__(self, title, img, parent=None):

        """ img should be an PIL image. """

        QtGui.QWidget.__init__(self, parent)

        self.ui = parent.ui

        layout = QtGui.QVBoxLayout()

        label = QtGui.QLabel()

        # directly wrting "image=ImageQt.ImageQt(img)" works fine on linux but crash on windows

        # we can find solution here - http://skilldrick.co.uk/2010/03/pyqt-pil-and-windows/

        if os.name == 'nt':

            QtImage = ImageQt.ImageQt(img)

            image = QtImage.copy()

        else:

            image = ImageQt.ImageQt(img)

        pixmap = QtGui.QPixmap.fromImage(image)

        label.setPixmap(pixmap)

        layout.addWidget(label)

        save_image = QtGui.QPushButton('Save Image')

        def save():

            if hasattr(img, 'tmp_file'):

                shutil.copy(img.tmp_file, path)

            else:

                try:

                    img.save(str(path))

                except Exception:

                    self.ui.display_error('Could not save image!')

            

        save_image.clicked.connect(save)

        layout.addWidget(save_image)

        self.setLayout(layout)

        self.setWindowTitle(title)

3. Problems about using libjpeg please refer to  "How did I solve frequent crashing of libjpeg?"

LSB in steganography

Steganography is very interesting. A simple method is LSB. We hide our information bit by bit in every pixel, and the number of bits and position of hidden bits are determined by our program. I've finished LSB for a long time but don't spread my implementation considering my poor programming ability. But impressed by my "elegant" code, I decided to share it in this blog. (In fact, I was inspired by someone else's code but couldn't think of where and who, please contact me if you have any opinions)

MATLAB makes people process image quite easily, so I used MATLAB to finish it. Here is my code :

% secret : matrix made of hidden infomation
% carrier: matrix made of pixel
% extract: matrix to store hidden information
% N: position, usually equaling 1
carrier = bitor( bitand(carrier, bitcmp(2^N-1, 8)), secret );
extract = uint8( bitand( 255, bitshift(carrier, 8-N)));

P.S. To implement a function, you might need dec2bin() mat2str() bin2dec() for support.