self-study lab - Python Tutorial Basic

Error and Issue of Pytorch Installation

After a couple of trial and error, here is the most promising way to install Pytorch in Ubuntu.

If you have already tried different ways to install Pytorch or you have  followed different posts to fix your problem, you need to first uninstall everything related to Pytorch.

To uninstall pytorch, here is what is suggested by soumith(https://github.com/pytorch/pytorch/issues/2036).

conda uninstall -y pytorch
pip uninstall torch
pip uninstall torch # yes run this twice

Now, reinstall pytorch. From the official website, we have different options. Here we recommand using conda. Go to the Pytorch website and choose your command. For example,

conda install pytorch torchvision -c soumith

Now, if you go to your IDE and try to import pytorch, you may get some error similiar to the error message below:

ImportError: /home/ruikun/anaconda/lib/python2.7/site-packages/torch/lib/libTHC.so.1: undefined symbol: THLongStorage_inferSizeN

To solve this problem, type:

conda uninstall libtorch

Summary

1. uninstall pytorch completely

2. reinstall pytorch via conda

3. conda uninstall libtorch

强烈推荐 - 信息简史 (图灵新知) ([美]詹姆斯·格雷克)

信息简史 (图灵新知) ([美]詹姆斯·格雷克)

“如果你对他说，有一台机器能削土豆，他会宣称这不可能；而如果你当面用机器削给他看，他又会宣称这玩意儿没用，因为它不能切菠萝。”[54]他们变得不再能把握重点之所在，总是挑发明的缺陷和不足，却忽视了其可能的用途。 “我们要怎么做才能摆脱巴贝奇先生和他的计算机器？”当时的英国首相罗伯特·皮尔（Robert Peel）在 1842年 8月写信给他的一位顾问咨询道，“可以肯定，就算得以完成，它对于科学也毫无价值……在我看来，它只是个非常昂贵的玩具。”而他也很容易在公职人员中找到对巴贝奇的不利之辞。其中最致命的抨击恐怕来自皇家天文学家乔治·比德尔·艾里（George Biddell Airy），这位思想僵化只认死理的人毫不含糊地说出了皮尔首相最想听到的话：“对于机器的有用性，我觉得他很可能是在白日做梦。”[55]就这样，皮尔的政府中止了这个项目。然而，巴贝奇的梦想却并未终止，并且早已转变方向，上升到了一个全新的高度。另外，他也已经结识了爱达·拜伦。 在伦敦劳瑟拱廊市集北端的河岸街上，坐落着由美国发明家雅各布·珀金斯（Jacob Perkins）创办的国家应用科学展馆。这是一个“寓教于乐”的地方，结合了玩具店和技术展



“如果你对他说，有一台机器能削土豆，他会宣称这不可能；而如果你当面用机器削给他看，他又会宣称这玩意儿没用，因为它不能切菠萝。”[54]他们变得不再能把握重点之所在，总是挑发明的缺陷和不足，却忽视了其可能的用途。

差异（differences）一词想必在威尔金斯的读者（尽管为数不多）看来是个奇怪的措辞。但这是有意之举，并且富有深意。威尔金斯正试图接近信息最纯粹、最一般的形式，而文字只不过是其中的一个特例：“因为我们必须意识到，一般说来，任何能够创造出有效的差异，可为某种感官所感知的东西，都足以作为表达思想的手段。”[75]这可以是“音调不同的两个钟”，或是“任何可见物，如火焰、狼烟等”，又或是号角、大炮或皮鼓。任何的差异都是一种二元选择，而任何的二元选择都可以拿来表达思想。在这里，在这份1641年不为人知的匿名专著中，信息论的核心思想浮出了人类思维的表面，却像土拨鼠一样看见了自己的影子，于是又沉没下去，消失长达三百多年



但这是有意之举，并且富有深意。威尔金斯正试图接近信息最纯粹、最一般的形式，而文字只不过是其中的一个特例：“因为我们必须意识到，一般说来，任何能够创造出有效的差异，可为某种感官所感知的东西，都足以作为表达思想的手段。”[75]这可以是“音调不同的两个钟”，或是“任何可见物，如火焰、狼烟等”，又或是号角、大炮或皮鼓。任何的差异都是一种二元选择，而任何的二元选择都可以拿来表达思想。在这里，在这份1641年不为人知的匿名专著中，信息论的核心思想浮出了人类思维的表面，却像土拨鼠一样看见了自己的影子，于是又沉没下去，消失长达三百多年。


差异（differences）一词想必在威尔金斯的读者（尽管为数不多）看来是个奇怪的措辞。但这是有意之举，并且富有深意。威尔金斯正试图接近信息最纯粹、最一般的形式，而文字只不过是其中的一个特例
：“因为我们必须意识到，一般说来，任何能够创造出有效的差异，可为某种感官所感知的东西，都足以作为表达思想的手段。”[75]这可以是“音调不同的两个钟”，或是
“任何可见物，如火焰、狼烟等”，又或是号角、大炮或皮鼓。任何的差异都是一种二元选择，而任何的二元选择都可以拿来表达思想。在这里，在这份1641
年不为人知的匿名专著中，信息论的核心思想浮出了人类思维的表面，却像土拨鼠一样看见了自己的影子，于是又沉没下去，消失长达三百多年。

表面上看来，这样一个命题是个错误的循环论证。但事实并非如此，因为一开始它仅仅是断言了某个定义良好的公式……是不可被证实的。只是后来（由于巧合），这个公式恰好就是表示该命题的那个公式。



并以此为题在1956年写了一篇仅有四段话的短文，特意进行提醒：“许多不同领域的科学家同人，为其浩大声势及其开辟的科学分析新思路所吸引，正纷纷将这些思想应用到自己研究的问题上去……尽管对于我们这些从事这个领域研究的人来说，这一波流行热潮固然让人欣喜和兴奋，但同时它其中也带有危险的元素。”[65]香农提醒人们，信息论的核心是数学。

“我很纳闷，你这年轻人是否意识到自己在摆弄的是些什么东西。” 要是你启动了一部这样的机械计算机，却忘了在出去吃午餐时关掉它，那会发生什么事呢？我来告诉你。它在美国就会像野兔在澳大利亚一样。在你算出 701 945 240乘以 879 030 546等于多少之前，这个国家的每个家庭就都有了一台自己的小计算机…… 香农先生，我无意批评你的这些实验，但坦率地讲，我对哪怕是一台计算机都丝毫没有兴趣。更别提要是有一堆计算机挤到我周围，又是乘又是除，又做其他什么它们擅长的事，那我一定会相当生气。 [73]


换句话说，H度量的是出人意料的程度。再换句话说，H是通过只允许回答是或否的问题来猜出一条未知讯息时所需问问题的平均数目。

这些远古的复制子并没有灭绝，毕竟它们是生存艺术的大师。不过现在你不会找到它们在海洋中四处漂流，因为它们很早便已经放弃了这种随波逐流的自由。它们如今聚集在一块块巨大的殖民地当中，藏身于笨重的机器人内，与外部世界相隔离，只是通过复杂的间接途径与其沟通，并通过远程控制对其进行操控。它们就在你我体内。它们创造了我们，包括身体和心智，而我们存在的终极目的只是为保全它们。这些复制子不断在完善着自己。现在它们被称为基因，而我们不过是它们的生存机器。[32] 可以想见，这样的说法势必会激怒某些不认为自己不过是机器人的人。1977年，史蒂芬·杰伊·古尔德就写道：“最近英国生物学家理查德·道金斯的说法，称基因才是自然选择的基本单位，而个体不过是它们暂时容身的皮囊，这令我十分不快。”[33]古尔德的同道者甚众。冈特·斯滕特说出了许多分子生物学家想说的话，称这位“三十六岁的动物行为专业的学生”的说法当中暗含着“万物有灵的这种古老的、前科学的传统”。[34] 尽管如此，道金斯这本精彩的著作还是引发了重大的转变，确立了一种对于基因的全新的、多层次的理解。乍看上去，自私的基因的思想不过是变换了视角，或像是开玩笑。塞缪尔·巴特勒在一百年前就说过（并且他并未宣称自己是第一个说这话的人），母鸡不过是



塞缪尔·巴特勒在一百年前就说过（并且他并未宣称自己是第一个说这话的人），母鸡不过是一只蛋用来制造另一只蛋的工具。

每种生物都有权以自己的方式“完成”自己的发育。蛋的做法可能看起来是绕了大弯，但那正是它自己的方式，我们人类没有多少理由对此说三道四。凭什么认为鸡比蛋更鲜活？又凭什么说是鸡生蛋而不是蛋生鸡？这些无法用哲学解释的问题，或许从另一个角度就可以解答，那就是考虑到人类的自负及其长久以来的、忽略一切不像自己的事物的习惯。

正如爱德华·奥斯本·威尔森在一百年后所说的，“人类中心主义是人类智识的一大残疾”。


但道金斯并没有动摇。他指出，基因说到底是关于差异。因此，他首先提出了一个与之密切相关的问题：是否存在对应阅读障碍症的基因呢？ 为了确认存在对应阅读的基因，我们所要做的只是发现对应不会阅读的基因，比如某个会造成大脑损伤、进而导致阅读障碍症的基因。这种阅读障碍症患者可能在一切方面都正常且聪颖，但就是不会阅读。如果这种阅读障碍症被证明会稳定遗传，大概遗传学家也不会感到特别惊讶。显然，在这个例子中，该基因只会在有着正规教育的环境中才会展现其效应。在史前环境中，该基因的效应可能就看不出来，或者会产生不同的效应，比如在原始人的遗传学家看来，该基因的效应就是无法辨别动物足迹…… 那么遵照遗传学术语的常规，同一基因座上的野生型基因（这个群体中其余个体都携有两份剂量的该基因），就可以恰当地被称为对应阅读的基因了。如果你反对这种说法，那么你也势必要反对我们在谈论孟德尔的豌豆时所用的诸如对应高茎的基因等说法，毕竟在这两种情况中，使用术语的逻辑是一样的。我们感兴趣的都是差异，并且差异也都只在某些特定环境下才会体现。至于为什么简单如一个基因的差异竟能导致如此复杂的效应，比如一个人是否能阅读，或鞋带系得好坏，理由大致如下：无论这个世界某个给定状态有多么复杂，该状态与另一种可替代状态之间的差异则可能是由某些极其简单的东西所导致的。[46]


思想保留了生物体的某些属性。与生物体一样，思想也倾向于维持其结构并繁衍生息；它们也可以融合、重组或分离其内容；事实上，它们还可以进化，而在这个过程中，选择淘汰必定扮演着重要角色。[3] 莫诺注意到，思想有“传



所有生命通过可自我复制的实体的生存差别［有的复制子存活，有的死亡］实现进化”。哪里有生命，哪里就一定有复制子。或许在其他世界里，复制子也能从硅基化学物质中产生——又或者完全与化学物质无关。 完全与化学物质无关的复制子，这是什么意思？“我认为，一种新的复制子最近已经在这个星球出现，”道金斯在他1976年的处女作结尾这样宣称，“它已经显露无遗。它虽然仍处于婴儿期，还在其原生汤里漂游，但它进化的速度已让古老的基因望尘莫及。”[5]这“汤”就是人类文化，传递载体就是语言，而产卵场就是大脑。

对此，惠勒至少还有所表述：“概率，正如时间一样，都是人类创造的概念，所以人类也必须对与之相关的晦涩之处负责。”[6]

相较于用物理方法生成随机数，冯·诺伊曼承认：“当然，任何考虑用算术方法来生成随机数的人都犯下了僭越之罪。因为，正如已经被多次指出的，不存在一个随机数这样的东西——有的只是生成随机数的方法，而一种严格的算术方法显然不属于其中之一。”[12]


我们呼吸的空气是怎样一番奇妙的混沌啊！……空气本身就是一座宏伟的图书馆，其中的书页上写满了世间男男女女的高声低语。在那里，变化多端但准确无误的字符记录下了从古至今逝者的叹息、未实现的誓言以及未履行的承诺。这些声音通过每个粒子的统一运动变得不朽，成为人类意志之善变的见证。[5]

（艾略特就接着写道：“我们在生存中失落的生活在哪里？/我们在知识中失落的智慧在哪里？/我们在信息中失落的知识又在哪里？”）

。诸如这样的警示，其中不免含有几分怀旧的气息，但无疑也包含了一个无可否认的真理：在追求知识的过程中，慢反而好。徜徉在弥漫着旧书气味的图书馆内，在堆满了典籍的书架之间探索，自有其回报。阅读（哪怕只是浏览）一本旧书所能获得的养分是从一次数据库搜索中无从得到的。耐心是美德，饕餮是罪过，无疑也适用于此。

许多研究者发现，信息过量不仅会导致困惑和挫败感，还会导致视野受限和不诚实行为。

Vim - C/C++ style indent

When edit C/C++ file with vim, sometime we have the problem that the close brace "}" cannot be indent correctly.

One solution is to use "=" utility in vim. In the normal mode, we can type gg=G. This will correct the indent issues in the current file. One of the inconvenience of this method is that you need to exit the insert mode and the edit process is interrupted.

Here is another solution. We still leverage the "=" utility in vim. In addition to that, we also need the command gi. it will send the cursor to the position where you were in the insert mode last time.

The full command is something looks like this:

    inoremap } <esc>a } ^]<s-v>=gi^]f}a

The goal is to map } in the insert mode and correct the possible indent issue and then back to the insert mode.

Initially, we are in the insert mode, <esc> let us exit the insert mode. (a }^] is to add an empty space before "}". It is not necessary for the purpose of correcting the indent issue.) <s-v> let us enter the visual mode, = will correct the indent issue of the current line. gi^] send us back to the position where the last edit occurs. f}a find the "}" and put the cursor after "}" and enter the insert mode.

Install OpenCV2

Step 1: Install OpenCV software

Follow the following steps mentioned in
https://gist.github.com/dynamicguy/3d1fce8dae65e765f7c4

Here we copy the commands:

# install dependencies
sudo apt-get update
sudo apt-get install -y build-essential
sudo apt-get install -y cmake
sudo apt-get install -y libgtk2.0-dev
sudo apt-get install -y pkg-config
sudo apt-get install -y python-numpy python-dev
sudo apt-get install -y libavcodec-dev libavformat-dev libswscale-dev
sudo apt-get install -y libjpeg-dev libpng-dev libtiff-dev libjasper-dev

sudo apt-get -qq install libopencv-dev build-essential checkinstall cmake pkg-config yasm libjpeg-dev libjasper-dev libavcodec-dev libavformat-dev libswscale-dev libdc1394-22-dev libxine-dev libgstreamer0.10-dev libgstreamer-plugins-base0.10-dev libv4l-dev python-dev python-numpy libtbb-dev libqt4-dev libgtk2.0-dev libmp3lame-dev libopencore-amrnb-dev libopencore-amrwb-dev libtheora-dev libvorbis-dev libxvidcore-dev x264 v4l-utils

# download opencv-2.4.11
wget http://downloads.sourceforge.net/project/opencvlibrary/opencv-unix/2.4.11/opencv-2.4.11.zip
unzip opencv-2.4.11.zip
cd opencv-2.4.11
mkdir release
cd release

# compile and install
cmake -G "Unix Makefiles" -D CMAKE_CXX_COMPILER=/usr/bin/g++ CMAKE_C_COMPILER=/usr/bin/gcc -D CMAKE_BUILD_TYPE=RELEASE -D CMAKE_INSTALL_PREFIX=/usr/local -D WITH_TBB=ON -D BUILD_NEW_PYTHON_SUPPORT=ON -D WITH_V4L=ON -D INSTALL_C_EXAMPLES=ON -D INSTALL_PYTHON_EXAMPLES=ON -D BUILD_EXAMPLES=ON -D WITH_QT=ON -D WITH_OPENGL=ON -D BUILD_FAT_JAVA_LIB=ON -D INSTALL_TO_MANGLED_PATHS=ON -D INSTALL_CREATE_DISTRIB=ON -D INSTALL_TESTS=ON -D ENABLE_FAST_MATH=ON -D WITH_IMAGEIO=ON -D BUILD_SHARED_LIBS=OFF -D WITH_GSTREAMER=ON ..
make all -j4 # 4 cores
sudo make install

We may get some errors when trying to install some of the libraries in command line. Since if highly depends on the system, we cannot give a general solution here. At the same time, many people have experiences the same problems and it is easy to find the solution through Google.

Step 2: Add Include path and Libraries.

If everything works, now we should have opencv2 installed. The next step is to configure our IDE. In my case, I use the eclipse. The instruction can be found here:
http://docs.opencv.org/2.4/doc/tutorials/introduction/linux_eclipse/linux_eclipse.html



Step 3: Test OpenCV

In this step, we use the first code example in the book Learning OpenCV (https://www.amazon.com/Learning-OpenCV-Computer-Vision-Library/dp/0596516134).

#include "highgui/highgui_c.h" // In the book, we have     
                               // #include "highgui.h"
                               // which is used in the old
                               // version of OpenCV


int main()
{
    IplImage* img = cvLoadImage(  "path-of-an-image-file"  );
    cvNamedWindow(  "Example1", CV_WINDOW_AUTOSIZE  );
    cvShowImage(  "Example1", img  );
    cvWaitKey( 0 );
    cvReleaseImage(  &img  );
    cvDestroyWindow(  "Example1"  );
 }

If we can see a new window of image pop up, we are all good.

Raspeberry Pi - Connect to Wifi and send ip information to email

There is problem with the current setting of my Raspeberry Pi. Every time when it is started, I need to log into my account and connect to wifi. In order to use ssh on my desktop, I also need the ip information of the Raspberry Pi.

To automate this part, we need to write some autostart program. Essentially, we want to write scripts that can be executed upon the start.

In ArchLinux, it can be doen through systemctl. The documentation can be found here:
https://wiki.archlinux.org/index.php/systemd

It takes a while to read through all the sections. Some Useful examples can be found here:
https://www.freedesktop.org/software/systemd/man/systemd.service.html

Here is what we need to do:

Step 1: Create the script that need to be executed upon the start.

In our case, we want to have something like

#!/bin/sh
ip link set wlan0 down
netctl start home_wifi
ifconfig | mail -v -s "IP information" user@gmail.com

After editing the script, make it executable: chmod + x your-script

Step 2: Create a new service

Go to the /etc/systemd/system folder and create a new file
my-autostart.service

[Unit]
Description=Send ip information through email

[Service]
ExecStart=/path/your-script

[Install]
WantedBy=multi-user.target

Save the edit and enable the service by
systemctl enable my-autostart.service

Caveate
Though it can work, the configuration of the service may not be completely correct. It is better to have more fine control on it.

Use Raspberry Pi as a server

With the help of gmail, we can use Raspberry Pi as a server. What we need are:

• mail: to send/reply email
• getmail: to get new email messages

It is possible to use plain python to send/receive email, but here it is more convenient to use the existing tools.

The request consists of

• subject which contains [request@me]. This is the request identifier
• request message included in the email message. Each request starts with @begin and ends with @end. Each line in between has the form of attributeName = attributeValue. For example, request = take-picture indicates that this request is asking raspberry-pi to take a picture.

 As mentioned in one of the previous post, we need the raspistill working on raspberry pi. Otherwise, the flow is:

1. Check new email messages by calling getmail -n
2. Parse the new mail messages if there is any and get the request object
3. Process the request

Here is the code:

import re
import subprocess
import os
from os import path
import hashlib
from datetime import datetime
import time
import subprocess
import shlex
import logging
import glob


_currDir = os.path.dirname( os.path.abspath( __file__ ) )

# setup logger
logging.basicConfig(level    = logging.DEBUG,
                    format   = '%(asctime)s %(name)s %(levelname)s %(message)s',
                    datefmt  = '%m-%d %H:%M',
                    filename = path.join(_currDir, "_send-emil.log"),
                    filemode ='w')

console = logging.StreamHandler()
console.setLevel(logging.INFO)
formatter = logging.Formatter('%(name)s: %(levelname)s %(message)s')
console.setFormatter(formatter)
logging.getLogger('').addHandler(console)

logging.info( "current directory: {}".format( _currDir ) )

# settings
REQUEST_IDENTIFIER = '[request@me]'
IDLE               = 'the parser is idle'
SUBJECT            = 'hitted subject'
BEGIN              = 'beginning-of-request'
IMG_FORMAT         = 'jpg'
RQ_TAKE_PICTURE    = 'take-picture'
ARG_NUM            = 'number'
MAIL_DIR           = '/home/meretciel/mail/new'
IMG_DIR            = '/home/meretciel/workspace/camera'
RECEIVERS          = [ 'meretciel.fr@gmail.com' ]


class EmailMessage( object ):
    _COMMAND_TEMPLATE_PLAIN      = r'mail -v -s "{subject}" {receiver}'
    _COMMAND_TEMPLATE_ATTACHMENT = r'mail -v -s "{subject}" {attachment} {receiver}'

    def __init__( self, subject = None, message = None, receiver = None, attachment = None ):
        self._subject    = subject
        self._message    = message
        self._receiver   = receiver
        self._attachment = attachment

    def _generateEmailCommand( self ):
        if isinstance( self._receiver, list ):
            receiver = ';'.join( self._receiver )
        else:
            receiver = self._receiver

        if not self._attachment:
            return EmailMessage._COMMAND_TEMPLATE_PLAIN.format( message = self._message, subject = self._subject, receiver = receiver )
            

        if isinstance( self._attachment, list):
            attachment = ' -a '.join( self._attachment )
            attachment = ' -a ' + attachment

        else:
            attachment = ' -a ' + self._attachment
        

        return EmailMessage._COMMAND_TEMPLATE_ATTACHMENT.format( message = self._message, subject = self._subject, receiver = receiver, attachment = attachment )

    
    def send( self ):
        ''' send email using mail command. We can also implement this in plain python '''
        p1 = subprocess.Popen( 
            shlex.split( r'echo {message}'.format( message = self._message ) ),
            stdout = subprocess.PIPE,
        )
        p2 = subprocess.Popen(
            shlex.split( self._generateEmailCommand() ),
            stdin = subprocess.PIPE
        )
        p2.communicate()

def constructCommand( s_command ):
    l = [ x for x in s_command.split(' ') if x != '' ]
    return l 

def executeCommand( s_command ):
    logging.info( "execute command: {}".format( s_command ) )
    subprocess.call( constructCommand( s_command ) )

def getEmail():
    executeCommand( "getmail -n" )




def checkNewEmail():
    ''' check if there is new ( unread ) emails. '''
    ls_output = subprocess.check_output( constructCommand( "ls -lt {}".format( MAIL_DIR ) ) )
    ls_output = str( ls_output, "utf-8" )
    logging.debug( "ls_output: {}".format( ls_output ) )
    ls_output = ls_output.split('\n')
    ls_output = ls_output[1:]   # the first line is total output
    newFiles  = [ x.split(' ')[-1] for x in ls_output ][::-1]
    newFiles  = [ path.join( MAIL_DIR, x ) for x in newFiles if x != '' ]

    return newFiles



def takePicture( number ):
    ''' ask raspiberry pi to take picture. '''
    time            = datetime.utcnow()
    baseFileName    = path.join( IMG_DIR, hashlib.sha224( str( time ).encode( "utf-8" ) ).hexdigest() )
    commandTemplate = r'/opt/vc/bin/raspistill -n -vf -w 640 -h 480 -e {IMG_FORMAT} {_} -o {baseFileName}%04d.{IMG_FORMAT}'.format( IMG_FORMAT = IMG_FORMAT, baseFileName = baseFileName, _ = "{time}")
    number          = min( 10, int( number ) )
    number          = max( number, 1 ) 

    if number == 1:
        command = commandTemplate.format( time = '' )
        imgFileNames = [ "{baseFileName}-001.{IMG_FORMAT}".format( baseFileName = baseFileName, IMG_FORMAT = IMG_FORMAT ) ]
    else:
        _tl           = 2000    # in milliseconds
        _totalTime    = _tl * ( number - 1)
        _time         = "-t {} -tl {}".format( _totalTime, _tl )
        command       = commandTemplate.format( time = _time )

        imgFileNames = ["{baseFileName}{num}.{IMG_FORMAT}".format( baseFileName = baseFileName, num=str(i).zfill(4), IMG_FORMAT=IMG_FORMAT ) for i in range(number)]

    try:
        executeCommand( command )
        return imgFileNames
    except Exception as e:
       logging.error("Error when taking picture") 
       return []


class Request( object ):

    def __init__( self ):
        self._requestName = None
        self._args = {}

    def load( self, attrName, value ):
        if attrName == 'request':
            self._requestName = value

        else:
            self._args.update( { attrName : value } )

    def process( self ):
        if self._requestName:
            logging.info( 'processing {}'.format( self ) )

            if self._requestName.lower() == RQ_TAKE_PICTURE:
                number = self._args.get( ARG_NUM, 1 )
                imgFiles = takePicture( number )
                logging.debug( '<2> image files : {}'.format( imgFiles ) )
                newEmail = EmailMessage( subject = 'New Images', message = '', receiver = RECEIVERS, attachment = imgFiles )
                newEmail.send()
                for fn in imgFiles:
                    logging.info( "removing the file: {}".format( fn ) )
                    os.remove( fn )


    def __repr__( self ):
        return "Request( name={}, args={} )".format( self._requestName, str( self._args ) )

def _parseEmail( f, state, requests ):
    if state == IDLE:
        line = f.readline()
        while line:
            if 'Subject' in line and REQUEST_IDENTIFIER in line:
                state = SUBJECT
                break
            line = f.readline()
        return line, f, state, requests

    if state == SUBJECT:
        line = f.readline()
        while line:
            if '@begin' in line:
                state = BEGIN
                break
            line = f.readline()
        return line, f, state, requests

    if state == BEGIN:
        pattern = r'(?P<attrName>\w+)\s*=\s*(?P<value>.+)'
        line = f.readline()
        request = Request()
        while line:
            if '@end' in line:
                requests.append( request )
                state = IDLE
                break

            res = re.search( pattern, line )
            if res:
                attrName  = res.group( 'attrName' )
                value     = res.group( 'value' )
                request.load( res.group( 'attrName' ), res.group( 'value' ) )

            line = f.readline()

        return line, f, state, requests
        



def parseEmail( msgFile ):
    logging.info("parsing email file {}".format( msgFile ) )
    with open( msgFile ) as f:
        state = IDLE
        line = '__start__'
        requests = []
        while line:
            logging.info( "processing line : {}".format( line ) )
            line, f, state, request = _parseEmail( f, state, requests )

        return requests

def removeNewMsgFiles( newMsgFiles ):

    for item in newMsgFiles:
        os.remove( item )


def getNewRequestFromEmail():
    getEmail()
    newMsgFiles = checkNewEmail()
    logging.debug(" <1> New messages : {}".format( str( newMsgFiles ) ) ) 
    requests = []
    for newMsgFile in newMsgFiles:
        requests.extend( parseEmail( newMsgFile ) )
    removeNewMsgFiles( newMsgFiles )
    return requests


if __name__ == '__main__':
    
    existingFiles = glob.glob( path.join( MAIL_DIR, r'*.alarmpi' ) )
    for f in existingFiles:
        os.remove( f )

    while True:
        logging.info( "waiting for request." )
        requests = getNewRequestFromEmail()
        for request in requests:
            request.process()

        time.sleep( 10. )

--END--

Ubuntu - French Accents with US Keyboard Layout

Original Source : https://askubuntu.com/questions/333227/type-french-accents-with-us-international-keyboard

 
 Use the keyboard layout called English (international with Alt Gr dead keys). (Do this by going to System Settings > Keyboard > Layout settings, and then add the layout you want with the + button on the bottom.)
This way, the keys to input the characters ', ", `, ~, and ^ become "dead keys" while you hold pressed Alt Gr. After releasing the key combination with Alt Gr, no characters will be print (hence "dead"), but inserting a letter afterwards will yield a variation of the base character of the letter, in accordance with the dead key you pressed before; for example, é, ë, è, ẽ, and ê for e (respectively). (If you have no explicit Alt Gr key, just use the rightmost Alt key.)
So in summary, to insert a symbol

1. press and hold Alt Gr,
2. press the modifier key / key combination; e.g., `,
3. release all keys pressed so far, and
4. press the base character key; e.g., E for è.

Note: You can also enter an acute accented vowel by just pressing Alt Gr + (vowel).
(Source: this post on french Ubuntu forums.)

More Effective C++ Reading Notes

More Effective C++

Item 1: Distinguish between pointers and references

• there is no such thing as a null reference.
• A reference must always refer to some object.
• The fact that there is no such thing as a null reference implies that it can be more efficient to use references than to use pointers. That's because there's no need to test the validity of a reference before using it.
• Another important difference between pointers and references is that pointers may be reassigned to refer to different objects. A reference, however, always refers to the object with which it is initialized.
• References, then, are the feature of choice when you know you have something to refer to, when you'll never want to refer to anything else, and when implementing operators whose syntactic requirements make the use of pointers undesirable. In all other cases, stick with pointers.

Item 2: Prefer C++ style casts

• C++ addresses the shortcomings of C-style casts by introducing four new cast operators
  
  • static_cast
    
  • const_cast
    
  • dynamic_cast
    
  • reinterpret_cast
    
• static_cast has basically the same power and meaning as the general-purpose C-style cast.
  
• static_cast can't remove constness from an expression, because another new cast, const_cast, is designed specifically to do that.
  
• const_cast is used to cast away the constness or volatileness of an expression. By using a const_cast, you emphasize (to both humans and compilers) that the only thing you want to change through the cast is the constness or volatileness of something.
  
• By far the most common use of const_cast is to cast away the constness of an object.
  
• dynamic_cast is used to perform safe casts down or across an inheritance hierarchy. That is, you use dynamic_cast to cast pointers or references to base class objects into pointers or references to derived or sibling base class objects in such a way that you can determine whether the casts succeeded. Failed casts are indicated by a null pointer (when casting pointers) or an exception (when casting references).
  
• dynamic_casts are restricted to helping you navigate inheritance hierarchies. They cannot be applied to types lacking virtual functions, , nor can they cast away constness.
  
• If you want to perform a cast on a type where inheritance is not involved, you probably want a static_cast. To cast constness away, you always want a const_cast.
  
• reinterpret_cast is used to perform type conversions whose result is nearly always implementation-defined. As a result, reinterpret_casts are rarely portable.
  

Item 3: Never treat arrays polymorphically

• The issue here is related to the pointer arithmetic. In the pointer arithemtic, the compiler needs to know the exact size of the object that the pointer points to.
• Polymorphism and pointer arithmetic simply don't mix.
• The language specification says the result of deleting an array of derived class objects through a base class pointer is undefined.
• Note that you're unlikely to make the mistake of treating an array polymorphically if you avoid having a concrete class inherit from another concrete class.

Item 4: Avoid gratuitous default constructors

• if a class lacks a default constructor, there are restrictions on how you can use that class.
• The first is the creation of arrays. There is, in general, no way to specify constructor arguments for objects in arrays.
• A general solution to this is to use an array of pointers instead of an array of objects. here are two disadvantages to this approach. First, you have to remember to delete all the objects pointed to by the array. If you forget, you have a resource leak. Second, the total amount of memory you need increases.
• The second problem with classes lacking default constructors is that they are ineligible for use with many template-based container classes. That's because it's a common requirement for such templates that the type used to instantiate the template provide a default constructor. This requirement almost always grows out of the fact that inside the template, an array of the template parameter type is being created.
• Virtual base classes lacking default constructors are a pain to work with. That's because the arguments for virtual base class constructors must be provided by the most derived class of the object being constructed. => ?

Item 5: Be wary of user-defined conversion functions.

• C++ allows compilers to perform implicit conversions between types. In honor of its C heritage, for example, the language allows silent conversions from char to int and from short to double.
• Two kinds of functions allow compilers to perform such conversions: single-argument constructors and implicit type conversion operators.
• A single-argument constructor is a constructor that may be called with only one argument. Such a constructor may declare a single parameter or it may declare multiple parameters, with each parameter after the first having a default value.
• An implicit type conversion operator is simply a member function with a strange-looking name: the word operator followed by a type specification. You aren't allowed to specify a type for the function's return value, because the type of the return value is basically just the name of the function.
  

• class Rational {
  public:
      ...
      operator double() const;
  };
  

• The fundamental problem is that such functions often end up being called when you neither want nor expect them to be. The result can be incorrect and unintuitive program behavior that is maddeningly difficult to diagnose.
• Compilers would try to find an acceptable sequence of implicit type conversions they could apply to make the call succeed.
• In most cases, the inconvenience of having to call conversion functions explicitly is more than compensated for by the fact that unintended functions can no longer be silently invoked.
• Implicit conversions via single-argument constructors are more difficult to eliminate. Furthermore, the problems these functions cause are in many cases worse than those arising from implicit type conversion operators.
• The easy way is to avail yourself of one of the newest C++ features, the explicit keyword. This feature was introduced specifically to address the problem of implicit type conversion, and its use is about as straightforward as can be. Constructors can be declared explicit, and if they are, compilers are prohibited from invoking them for purposes of implicit type conversion. Explicit conversions are still legal.
• There are complicated rules governing which sequences of implicit type conversions are legitimate and which are not. One of those rules is that no sequence of conversions is allowed to contain more than one user-defined conversion (i.e., a call to a single-argument constructor or an implicit type conversion operator).
• By constructing your classes properly, you can take advantage of this rule so that the object constructions you want to allow are legal, but the implicit conversions you don't want to allow are illegal.
• proxy classes technique.

Item 6: Distinguish between prefix and postfix forms of increment and decrement operators

• There was a syntactic problem, however, and that was that overloaded functions are differentiated on the basis of the parameter types they take, but neither prefix nor postfix increment or decrement takes an argument. To surmount this linguistic pothole, it was decreed that postfix forms take an int argument, and compilers silently pass 0 as that int when those functions are called.
• More important to get used to, however, is this: the prefix and postfix forms of these operators return different types. In particular, prefix forms return a reference, postfix forms return a const object.
• The prefix form of the increment operator is sometimes called "increment and fetch," while the postfix form is often known as "fetch and increment."
• That principle is that postfix increment and decrement should be implemented in terms of their prefix counterparts.
• When dealing with user-defined types, prefix increment should be used whenever possible, because it's inherently more efficient.
• If you've ever wondered if it makes sense to have functions return const objects, now you know: sometimes it does, and postfix increment and decrement are examples.

Item 7: Never overload &&, ||, or ,.

• It you overload the operator && and ||, you are changing the rules of the game quite radically, because you are replacing short-circuit semantics with function call semantics.
• when a function call is made, all parameters must be evaluated, so when calling the functions operator&& and operator||, both parameters are evaluated. There is, in other words, no short circuit.
• the language specification leaves undefined the order of evaluation of parameters to a function call, so there is no way of knowing whether expression1 or expression2 will be evaluated first. This stands in stark contrast to short-circuit evaluation, which always evaluates its arguments in left-to-right order.
• An expression containing a comma is evaluated by first evaluating the part of the expression to the left of the comma, then evaluating the expression to the right of the comma; the result of the overall comma expression is the value of the expression on the right.
• If you write operator, as a non-member function, you'll never be able to guarantee that the left-hand expression is evaluated before the right-hand expression, because both expressions will be passed as arguments in a function call (to operator,).
• That leaves only the possibility of writing operator, as a member function. Even here you can't rely on the left-hand operand to the comma operator being evaluated first, because compilers are not constrained to do things that way.
• Hence, you can't overload the comma operator and also guarantee it will behave the way it's supposed to. It therefore seems imprudent to overload it at all.

Item 8: Understand the different meanings of new and delete

• placement new: There are times when you really want to call a constructor directly. Invoking a constructor on an existing object makes no sense, because constructors initialize objects, and an object can only be initialized given its first value once. But occasionally you have some raw memory that's already been allocated, and you need to construct an object in the memory you have. A special version of operator new called placement new allows you to do it.
• An example of how ploacement new might be used:

• class Widget {
  public:
      Widget(int widgetSize);
      ...
  };
  
  Widget * constructWidgetInBuffer(void *buffer, int widgetSize) {
      return new (buffer) Widget(widgetSize);
  }
  

• If we step back from placement new for a moment, we'll see that the relationship between the new operator and operator new, though you want to create an object on the heap, use the new operator. It both allocates memory and calls a constructor for the object. If you only want to allocate memory, call operator new; no constructor will be called. If you want to customize the memory allocation that takes place when heap objects are created, write your own version of operator new and use the new operator; it will automatically invoke your custom version of operator new. If you want to construct an object in memory you've already got a pointer to, use placement new. => ??? 这边有点没有看的太明白。
  

• The new operator calls a function to perform the requisite memory allocation, and you can rewrite or overload that function to change its behavior. The name of the function the new operator calls to allocate memory is operator new.

• Operator new is a function that allocates raw memory -- at least conceptually, it's not much different from malloc.The new operator is what you normally use to create an object from the free store().

• One implication of this is that if you want to deal only with raw, uninitialized memory, you should bypass the new and delete operators entirely. Instead, you should call operator new to get the memory and operator delete to return it to the system:
  

• // The following code is the C++ equivalent of caling malloc and free.
  void *buffer = operator new(20 * sizeof(char)); // allocate enough memory to hold 50 chars
  operator delete(buffer) // deallocate the memory
  

• If you use placement new to create an object in some memory, you should avoid using the delete operator on that memory. That's because the delete operator calls operator delete to deallocate the memory, but the memory containing the object wasn't allocated by operator new in the first place; placement new just returned the pointer that was passed to it. Who knows where that pointer came from? Instead, you should undo the effect of the constructor by explicitly calling the object's destructor.

Item 9: Use destructors to prevent resource leaks

• local objects are always destroyed when leaving a function, regardless of how that function is exited. (The only exception to this rule is when you call longjmp, and this shortcoming of longjmp is the primary reason why C++ has support for exceptions in the first place.)
• Objects that act like pointers, but do more, are called smart pointers.
• The standard C++ library contains a class template called auto_ptr that does just what we want. Each auto_ptr class takes a pointer to a heap object in its constructor and deletes that object in its destructor.
• use auto_ptr objects instead of raw pointers, and you won't have to worry about heap objects not being deleted, not even when exceptions are thrown.
• By adhering to the rule that resources should be encapsulated inside objects, you can usually avoid resource leaks in the presence of exceptions.

Item 10: Prevent resource leaks in constructors

• C++ destroys only fully constructed objects, and an object isn't fully constructed until its constructor has run to completion.
• partially constructed objects aren't automatically destroyed.
• Because C++ won't clean up after objects that throw exceptions during construction, you must design your constructors so that they clean up after themselves. Often, this involves simply catching all possible exceptions, executing some cleanup code, then rethrowing the exception so it continues to propagate.
• if you replace pointer class members with their corresponding auto_ptr objects, you fortify your constructors against resource leaks in the presence of exceptions, you eliminate the need to manually deallocate resources in destructors, and you allow const member pointers to be handled in the same graceful fashion as non-const pointers.
• Dealing with the possibility of exceptions during construction can be tricky, but auto_ptr (and auto_ptr-like classes) can eliminate most of the drudgery. Their use leaves behind code that's not only easy to understand, it's robust in the face of exceptions, too.
• if a constructor throws an exception, what destructors are run?
  
  • Destructors of all the objects completely created in that scope.
• Does it make any difference if the exception is during the initialization list or the body?
  
  • All completed objects will be destructed. If constructor was never completely called object was never constructed and hence cannot be destructed.
• what about inheritance and members? Presumably all completed constructions get destructed. If only some members are constructed, do only those get destructed? If there is multiple inheritance, do all completed constructors get destructed? Does virtual inheritance change anything?
  
  • All completed constructions do get destructed. Yes, only the completely created objects get destructed.

Item 11: Prevent exceptions from leaving destructors

• There are two situations in which a destructor is called.
  • The first is when an object is destroyed under "normal" conditions, e.g., when it goes out of scope or is explicitly deleted.
  • The second is when an object is destroyed by the exception-handling mechanism during the stack-unwinding part of exception propagation.
• an exception may or may not be active when a destructor is invoked. Regrettably, there is no way to distinguish between these conditions from inside a destructor. As a result, you must write your destructors under the conservative assumption that an exception is active, because if control leaves a destructor due to an exception while another exception is active, C++ calls the terminate function.
• One way to handle this is to add a try block in the destructor:

• Session::~Session() {
      try {
          logDestruction(this);
      } catch (...) {}
  }
  

• There is a second reason why it's bad practice to allow exceptions to propagate out of destructors. If an exception is thrown from a destructor and is not caught there, that destructor won't run to completion. (It will stop at the point where the exception is thrown.)

Item 12: Understand how throwing an exception differs from passing a parameter or calling a virtual function

• This difference grows out of the fact that when you call a function, control eventually returns to the call site (unless the function fails to return), but when you throw an exception, control does not return to the throw site.
• C++ specifies that an object thrown as an exception is always copied. The reason behind this is that when the control leave the original scopr, the destructor of local varialbes will be called. That's why the local object need be to copied.
• This copying occurs even if the object being thrown is not in danger of being destroyed.
• When an object is copied for use as an exception, the copying is performed by the object's copy constructor. This copy constructor is the one in the class corresponding to the object's static type, not its dynamic type.
• This behavior may not be what you want, but it's consistent with all other cases in which C++ copies objects. Copying is always based on an object's static type (but see Item 25 for a technique that lets you make copies on the basis of an object's dynamic type).
• syntax to rethrow the current exception, because there's no chance that that will change the type of the exception being propagated. Furthermore, it's more efficient, because there's no need to generate a new exception object.
• About all you need to remember is not to throw a pointer to a local object, because that local object will be destroyed when the exception leaves the local object's scope. The catch clause would then be initialized with a pointer to an object that had already been destroyed. This is the behavior the mandatory copying rule is designed to avoid.
• In catch claus, there is no implicit type conversion.
• Two kinds of conversions are applied when matching exceptions to catch clauses.The first is inheritance-based conversions. A catch clause for base class exceptions is allowed to handle exceptions of derived class types, too. The second type of allowed conversion is from a typed to an untyped pointer, so a catch clause taking a const void* pointer will catch an exception of any pointer type.

• catch (runtime_error) ...               // can catch errors of type
  catch (runtime_error&) ...              // runtime_error,
  catch (const runtime_error&) ...        // range_error, or
                                          // overflow_error
  

• The final difference between passing a parameter and propagating an exception is that catch clauses are always tried in the order of their appearance.

Item 13: Catch exceptions by reference

• Catch-by-value gives rise to the specter of the slicing problem, whereby derived class exception objects caught as base class exceptions have their derivedness "sliced off." Such "sliced" objects are base class objects: they lack derived class data members, and when virtual functions are called on them, they resolve to virtual functions of the base class.
• If you catch by reference, you sidestep questions about object deletion that leave you damned if you do and damned if you don't; you avoid slicing exception objects; you retain the ability to catch standard exceptions; and you limit the number of times exception objects need to be copied. So what are you waiting for? Catch exceptions by reference!

Item 14 : Use exception specifications judiciously

• The default behavior for unexpected is to call terminate, and the default behavior for terminate is to call abort, so the default behavior for a program with a violated exception specification is to halt. Local variables in active stack frames are not destroyed, because abort shuts down program execution without performing such cleanup. A violated exception specification is therefore a cataclysmic thing, something that should almost never happen.
• Unfortunately, it's easy to write functions that make this terrible thing occur. Compilers only partially check exception usage for consistency with exception specifications. What they do not check for is a call to a function that might violate the exception specification of the function making the call.
• This is a specific example of a more general problem, namely, that there is no way to know anything about the exceptions thrown by a template's type parameters. We can almost never provide a meaningful exception specification for a template, because templates almost invariably use their type parameter in some way. The conclusion? Templates and exception specifications don't mix.
• If preventing unexpected exceptions isn't practical, you can exploit the fact that C++ allows you to replace unexpected exceptions with exceptions of a different type.

• class UnexpectedException {};          // all unexpected exception
                                         // objects will be replaced
                                         // by objects of this type
  
  void convertUnexpected()               // function to call if
  {                                      // an unexpected exception
    throw UnexpectedException();         // is thrown
  }
  
  // make it happen by replacing the default unexpected function with convertUnexpected: 
  set_unexpected(convertUnexpected);
  

• Another way to translate unexpected exceptions into a well known type is to rely on the fact that if the unexpected function's replacement rethrows the current exception, that exception will be replaced by a new exception of the standard type bad_exception.

• void convertUnexpected()          // function to call if
  {                                 // an unexpected exception
    throw;                          // is thrown; just rethrow
  }                                 // the current exception
  
  set_unexpected(convertUnexpected);
                                    // install convertUnexpected
                                    // as the unexpected
                                    // replacement
  

• It's important to keep a balanced view of exception specifications. They provide excellent documentation on the kinds of exceptions a function is expected to throw, and for situations in which violating an exception specification is so dire as to justify immediate program termination, they offer that behavior by default. At the same time, they are only partly checked by compilers and they are easy to violate inadvertently. Furthermore, they can prevent high-level exception handlers from dealing with unexpected exceptions, even when they know how to. That being the case, exception specifications are a tool to be applied judiciously. Before adding them to your functions, consider whether the behavior they impart to your software is really the behavior you want.

Item 15 : Understand the costs of exception handling

• Let us begin with the things you pay for even if you never use any exception-handling features. You pay for the space used by the data structures needed to keep track of which objects are fully constructed (see Item 10), and you pay for the time needed to keep these data structures up to date. These costs are typically quite modest.
• In theory, you don't have a choice about these costs: exceptions are part of C++, compilers have to support them, and that's that.
• A second cost of exception-handling arises from try blocks, and you pay it whenever you use one, i.e., whenever you decide you want to be able to catch exceptions. As a rough estimate, expect your overall code size to increase by 5-10% and your runtime to go up by a similar amount if you use try blocks. This assumes no exceptions are thrown; what we're discussing here is just the cost of having try blocks in your programs. To minimize this cost, you should avoid unnecessary try blocks.
• Compilers tend to generate code for exception specifications much as they do for try blocks, so an exception specification generally incurs about the same cost as a try block.
• Compared to a normal function return, returning from a function by throwing an exception may be as much as three orders of magnitude slower.
• The prudent course of action is to be aware of the costs described in this item, but not to take the numbers very seriously.
• To minimize your exception-related costs, compile without support for exceptions when that is feasible; limit your use of try blocks and exception specifications to those locations where you honestly need them; and throw exceptions only under conditions that are truly exceptional. If you still have performance problems, profile your software (see Item 16) to determine if exception support is a contributing factor. If it is, consider switching to different compilers, ones that provide more efficient implementations of C++'s exception-handling features.

Item 16: Remember the 80-20 rule

• the overall performance of your software is almost always determined by a small part of its constituent code.
• Remember that a profiler can only tell you how a program behaved on a particular run (or set of runs), so if you profile a program using input data that is unrepresentative, you're going to get back a profile that is equally unrepresentative. That, in turn, is likely to lead to you to optimize your software's behavior for uncommon uses, and the overall impact on common uses may even be negative.
• The best way to guard against these kinds of pathological results is to profile your software using as many data sets as possible.

Item 17: Consider using lazy evaluation

• declare the pointer fields mutable, which means they can be modified inside any member function, even inside const member functions.
  
• The mutable keyword is a relatively recent addition to C++, so it's possible your vendors don't yet support it. If not, you'll need to find another way to convince your compilers to let you modify data members inside const member functions. One workable strategy is the "fake this" approach, whereby you create a pointer-to-non-const that points to the same object as this does. When you want to modify a data member, you access it through the "fake this" pointer. => const_cast<yourObject>(this)
  

Item 18: Amortize the cost of expected computations

• That's because it's faster to read a big chunk once than to read two or three small chunks at different times. Furthermore, experience has shown that if data in one place is requested, it's quite common to want nearby data, too. This is the infamous locality of reference phenomenon, and systems designers rely on it to justify disk caches, memory caches for both instructions and data, and instruction prefetches.
• Lazy evaluation is a technique for improving the efficiency of programs when you must support operations whose results are not always needed. Over-eager evaluation is a technique for improving the efficiency of programs when you must support operations whose results are almost always needed or whose results are often needed more than once. Both are more difficult to implement than run-of-the-mill eager evaluation, but both can yield significant performance improvements in programs whose behavioral characteristics justify the extra programming effort.

Item 19: Understand the origin of temporary objects

• True temporary objects in C++ are invisible - they don't appear in your source code. They arise whenever a non-heap object is created but not named.
• Such unnamed objects usually arise in one of two situations: when implicit type conversions are applied to make function calls succeed and when functions return objects.
• These conversions occur only when passing objects by value or when passing to a reference-to-const parameter. They do not occur when passing an object to a reference-to-non-const parameter.
• Implicit type conversion for references-to-non-const objects, then, would allow temporary objects to be changed when programmers expected non-temporary objects to be modified. That's why the language prohibits the generation of temporaries for non-const reference parameters. Reference-to-const parameters don't suffer from this problem, because such parameters, by virtue of being const, can't be changed.
• => take a look at the move function and return-by-reference. Looks like they are relevant to this item

Item 20: Facilitate the return value optimization

• It is frequently possible to write functions that return objects in such a way that compilers can eliminate the cost of the temporaries. The trick is to return constructor arguments instead of objects, and you can do it like this:
  

• // an efficient and correct way to implement a
  // function that returns an object
  const Rational operator*(const Rational& lhs,
                           const Rational& rhs)
  {
    return Rational(lhs.numerator() * rhs.numerator(),
                    lhs.denominator() * rhs.denominator());
  }
  

• It even has a name: the return value optimization. In fact, the existence of a name for this optimization may explain why it's so widely available.

Item 21: Overload to avoid implicit type conversions

• Reasonable or not, there are rules to this C++ game, and one of them is that every overloaded operator must take at least one argument of a user-defined type. int isn't a user-defined type, so we can't overload an operator taking only arguments of that type.
• Overloading to avoid temporaries isn't limited to operator functions.
• the folloing code is used to avoid the creation of a temporary object of type UPInt.

• const UPInt operator+(const UPInt& lhs,      // add UPInt
                        const UPInt& rhs);     // and UPInt
  
  const UPInt operator+(const UPInt& lhs,      // add UPInt
                        int rhs);              // and int
  
  const UPInt operator+(int lhs,               // add int and
                        const UPInt& rhs);     // UPInt
  
  UPInt upi1, upi2;
  
  ...
  
  UPInt upi3 = upi1 + upi2;                    // fine, no temporary for
                                               // upi1 or upi2
  
  upi3 = upi1 + 10;                            // fine, no temporary for
                                               // upi1 or 10
  
  upi3 = 10 + upi2;                            // fine, no temporary for
                                               // 10 or upi2
  

Item 22: Consider using op= instead of stand-alone op

• This one turns out to be a very interesting topic. It involves named object, unnamed objects and compiler optimizations.
• If you don't mind putting all stand-alone operators at global scope, you can use templates to eliminate the need to write the stand-alone functions:

• // code 22.1
  template<class T>
  const T operator+(const T& lhs, const T& rhs)
  {
    return T(lhs) += rhs;     
  }                
  
  template<class T>
  const T operator-(const T& lhs, const T& rhs)
  {
    return T(lhs) -= rhs;  
  }
  

• In the above code, we return a unnamed object. Sometimes, we tend to do the following. It looks almost the same, but there is a crutial difference here.
  

• // code 22.2
  template<class T>
  const T operator+(const T& lhs, const T& rhs)
  {
    T result(lhs);                             // copy lhs into result
    return result += rhs;                      // add rhs to it and return
  }
  

• This second template contains a named object, result. The fact that this object is named means that the return value optimization (see Item 20) was, until relatively recently, unavailable for this implementation of operator+. The first implementation has always been eligible for the return value optimization, so the odds may be better that the compilers you use will generate optimized code for it.

Item 23: Consider alternative libraries

• Because different libraries embody different design decisions regarding efficiency, extensibility, portability, type safety, and other issues, you can sometimes significantly improve the efficiency of your software by switching to libraries whose designers gave more weight to performance considerations than to other factors.

Item 24: Understand the costs of virtual functions, multiple inheritance, virtual base classes, and RTTI

• When a virtual function is called, the code executed must correspond to the dynamic type of the object on which the function is invoked; the type of the pointer or reference to the object is immaterial. How can compilers provide this behavior efficiently? Most implementations use virtual tables and virtual table pointers. Virtual tables and virtual table pointers are commonly referred to as vtbls and vptrs, respectively.
  
• Virtual functions per se are not usually a performance bottleneck.
  
• The real runtime cost of virtual functions has to do with their interaction with inlining. For all practical purposes, virtual functions aren't inlined. That's because "inline" means "during compilation, replace the call site with the body of the called function," but "virtual" means "wait until runtime to see which function is called."
  

Item 25: Virtualizing constructors and non-member functions

• Because it creates new objects, it acts much like a constructor, but because it can create different types of objects, we call it a virtual constructor. A virtual constructor is a function that creates different types of objects depending on the input it is given.
• A particular kind of virtual constructor - the virtual copy constructor - is also widely useful.
• A virtual copy constructor returns a pointer to a new copy of the object invoking the function. Because of this behavior, virtual copy constructors are typically given names like copySelf, cloneSelf, or, as shown below, just plain clone.

• lass NLComponent {
  public:
    // declaration of virtual copy constructor
    virtual NLComponent * clone() const = 0;
    ...
  
  };
  
  class TextBlock: public NLComponent {
  public:
    virtual TextBlock * clone() const         // virtual copy
    { return new TextBlock(*this); }          // constructor
    ...
  
  };
  
  class Graphic: public NLComponent {
  public:
    virtual Graphic * clone() const            // virtual copy
    { return new Graphic(*this); }             // constructor
    ...
  
  };
  

• Notice that the above implementation takes advantage of a relaxation in the rules for virtual function return types that was adopted relatively recently.
• No longer must a derived class's redefinition of a base class's virtual function declare the same return type. Instead, if the function's return type is a pointer (or a reference) to a base class, the derived class's function may return a pointer (or reference) to a class derived from that base class.
• This opens no holes in C++'s type system, and it makes it possible to accurately declare functions such as virtual copy constructors.
• Making non-member functions act virtual. This is also a very interesting topic. In the example below, we dethe operator<< so that its behavior depens on the dynamic type of the argument.

• class NLComponent {
  public:
    virtual ostream& print(ostream& s) const = 0;
  ...
  };
  class TextBlock: public NLComponent {
  public:
    virtual ostream& print(ostream& s) const;
  ...
  };
  class Graphic: public NLComponent {
  public:
    virtual ostream& print(ostream& s) const;
  ...
  };
  inline
  ostream& operator<<(ostream& s, const NLComponent& c)
  {
    return c.print(s);
  }
  

Item 26: Limiting the number of objects of a class

• Each time an object is instantiated, we know one thing for sure: a constructor will be called. That being the case, the easiest way to prevent objects of a particular class from being created is to declare the constructors of that class private.
• The code blow shows how to make sure there is only one Priter object.

• class PrintJob;                              // forward declaration
                                               // see Item E34
  class Printer {
  public:
    void submitJob(const PrintJob& job);
    void reset();
    void performSelfTest();
  ...
  friend Printer& thePrinter();
  private:
    Printer();
    Printer(const Printer& rhs);
  ...
  };
  Printer& thePrinter()
  {
    static Printer p;                          // the single printer object
    return p;
  }
  

• Now if we do not want to put the thePrinter in the global namespace, here is the solution. It is the standard singleton implementation.
  

• class Printer {
  public:
    static Printer& thePrinter();
    ...
  
  private:
    Printer();
    Printer(const Printer& rhs);
    ...
  
  };
  
  Printer& Printer::thePrinter()
  {
    static Printer p;
    return p;
  }
  
  // Clients must now be a bit wordier when they refer to the printer: 
  
  Printer::thePrinter().reset();
  Printer::thePrinter().submitJob(buffer);
  

• Another approach is to move Printer and thePrinter out of the global scope and into a namespace.
• There are two subtleties in the implementation of thePrinter that are worth exploring.
• First, it's important that the single Printer object be static in a function and not in a class. An object that's static in a class is, for all intents and purposes, always constructed (and destructed), even if it's never used. In contrast, an object that's static in a function is created the first time through the function, so if the function is never called, the object is never created.
• Consider for a moment why you'd declare an object to be static. It's usually because you want only a single copy of that object, right? Now consider what inline means. Conceptually, it means compilers should replace each call to the function with a copy of the function body, but for non-member functions, it also means something else. It means the functions in question have internal linkage.
• functions with internal linkage may be duplicated within a program (i.e., the object code for the program may contain more than one copy of each function with internal linkage), and this duplication includes static objects contained within the functions. The result? If you create an inline non-member function containing a local static object, you may end up with more than one copy of the static object in your program! So don't create inline non-member functions that contain local static data.
• Here is a very interesting application of the template. It makes me think of the meta class in Python. The motiviation is that we do not want to repeat the code for counting the number of instance of a class. One of the approaches to use Template. Note that in the code below, we do not have BeingCounted appear in the definition.

• template<class BeingCounted>
  class Counted {
  public:
    class TooManyObjects{};                     // for throwing exceptions
  
    static int objectCount() { return numObjects; }
  
  protected:
    Counted();
    Counted(const Counted& rhs);
  
    ~Counted() { --numObjects; }
  
  private:
    static int numObjects;
    static const size_t maxObjects;
  
    void init();                                // to avoid ctor code
  };                                            // duplication
  
  template<class BeingCounted>
  Counted<BeingCounted>::Counted()
  { init(); }
  
  template<class BeingCounted>
  Counted<BeingCounted>::Counted(const Counted<BeingCounted>&)
  { init(); }
  
  template<class BeingCounted>
  void Counted<BeingCounted>::init()
  {
    if (numObjects >= maxObjects) throw TooManyObjects();
    ++numObjects;
  }
  

• If we want to count the number of instance of a class, we can define that class by inheriting the Counted class. For eample, we can do the following

• class Printer: private Counted<Printer> {
  public:
    // pseudo-constructors
    static Printer * makePrinter();
    static Printer * makePrinter(const Printer& rhs);
  
    ~Printer();
  
    void submitJob(const PrintJob& job);
    void reset();
    void performSelfTest();
    ...
  
    using Counted<Printer>::objectCount;     // see below
    using Counted<Printer>::TooManyObjects;  // see below
  
  private:
    Printer();
    Printer(const Printer& rhs);
  };
  
  // make objectCount public
  
  class Printer: private Counted<Printer> {
  public:
    ...
    using Counted<Printer>::objectCount; // make this function
                                         // public for clients
    ...                                  // of Printer
  };
  
  // or 
  
  class Printer: private Counted<Printer> {
  public:
    ...
    Counted<Printer>::objectCount;       // make objectCount
                                         // public in Printer
    ...
  
  };
  

• In order to specify the maxObjects, we can do it in the implementation file.

• const size_t Counted<Printer>::maxObjects = 10;
  const size_t Counted<FileDescriptor>::maxObjects = 16;
  

• What will happen if these authors forget to provide a suitable definition for maxObjects? Simple: they'll get an error during linking, because maxObjects will be undefined. Provided we've adequately documented this requirement for clients of Counted, they can then say "Duh" to themselves and go back and add the requisite initialization.

Item 27: Requiring or prohibiting heap-based objects

• TBD: Requiring heap-based object. => need to review
• The outlook of prohibiting heap-based objects is a bit better. What we can do is to declare operator new and operator delete private. In this way, clients will not be able to use the new operator.
• The bond between operator new and operator delete is stronger than many people think. For information on a rarely-understood aspect of their relationship, turn to the sidebar in my article on counting objects. => Counting Objects in C++ by Scott Meyers
• Interestingly, declaring operator new private often also prevents UPNumber objects from being instantiated as base class parts of heap-based derived class objects. That's because operator new and operator delete are inherited, so if these functions aren't declared public in a derived class, that class inherits the private versions declared in its base(s).
• If the derived class declares an operator new of its own, that function will be called when allocating derived class objects on the heap, and a different way will have to be found to prevent UPNumber base class parts from winding up there. Similarly, the fact that UPNumber's operator new is private has no effect on attempts to allocate objects containing UPNumber objects as members.

Item 28: Smart pointers

• When you use smart pointers in place of C++'s built-in pointers (i.e., dumb pointers), you gain control over the following aspects of pointer behavior
  • Construction adn destruction
  • Copying and assignement
  • Dereferencing
• The template code for smart pointers

• template<class T>                    // template for smart
  class SmartPtr {                     // pointer objects
  public:
    SmartPtr(T* realPtr = 0);          // create a smart ptr to an
                                       // obj given a dumb ptr to
                                       // it; uninitialized ptrs
                                       // default to 0 (null)
  
    SmartPtr(const SmartPtr& rhs);     // copy a smart ptr
  
    ~SmartPtr();                       // destroy a smart ptr
  
    // make an assignment to a smart ptr
    SmartPtr& operator=(const SmartPtr& rhs);
  
    T* operator->() const;             // dereference a smart ptr
                                       // to get at a member of
                                       // what it points to
  
    T& operator*() const;              // dereference a smart ptr
  
  private:
    T *pointee;                        // what the smart ptr
  };                                   // points to
  

• => auto_ptr template from the standard C++ library
• One of the difficulties in the smart pointer design is the ownership of the object.
• A flexible solution was adopted for the auto_ptr classes: object ownership is transferred when an auto_ptr is copied or assigned.
• Here is how the copy constructor and assignment operatio are implemented for auto_ptr.

• template<class T>
  auto_ptr<T>::auto_ptr(auto_ptr<T>& rhs)
  {
    pointee = rhs.pointee;             // transfer ownership of
                                       // *pointee to *this
  
    rhs.pointee = 0;                   // rhs no longer owns
  }                                    // anything
  
  template<class T>
  auto_ptr<T>& auto_ptr<T>::operator=(auto_ptr<T>& rhs)
  {
    if (this == &rhs)                  // do nothing if this
      return *this;                    // object is being assigned
                                       // to itself
  
    delete pointee;                    // delete currently owned
                                       // object
  
    pointee = rhs.pointee;             // transfer ownership of
    rhs.pointee = 0;                   // *pointee from rhs to *this
  
    return *this;
  }
  

• Because object ownership is transferred when auto_ptr's copy constructor is called, passing auto_ptrs by value is often a very bad idea.
• Now let us examine the operator->

• // The following statement 
  
  pt->displayEditDialog();
  
  // is interpreted by compilers as: 
  
  (pt.operator->())->displayEditDialog();
  

• That means that whatever operator-> returns, it must be legal to apply the member-selection operator (->) to it. There are thus only two things operator-> can return: a dumb pointer to an object or another smart pointer object. Most of the time, you'll want to return an ordinary dumb pointer.
• One of the things we cannot do with smart pointer is find out if a smart pointer is null.
• Implicit conversion to null pointer.

• template<class T>
  class SmartPtr {
  public:
    ...
    operator void*();                  // returns 0 if the smart
    ...                                // ptr is null, nonzero
  };                                   // otherwise
  

• The bottom line is simple: don't provide implicit conversion operators to dumb pointers unless there is a compelling reason to do so.
• Another problem with the smart pointer is how to handle the inheritance. In the example, we have a base class called MusicProduct, and two derived classes Cassette and CD. Now think about it a little while, the SmartPtr<MusicProduct>, SmarPtr<Cassette> and SmartPtr<CD> are three different classes. In other words, we lose the inheritence relationship. This is not good.
• One of the solutions to the above problem is to use member function tempaltes.

• template<class T>                    // template class for smart
  class SmartPtr {                     // pointers-to-T objects
  public:
    SmartPtr(T* realPtr = 0);
  
    T* operator->() const;
    T& operator*() const;
  
    template<class newType>             // template function for
    operator SmartPtr<newType>()        // implicit conversion ops.
    {
      return SmartPtr<newType>(pointee);
    }
  
    ...
  };
  

• Implementing smart pointer conversions through member templates has two additional drawbacks. First, support for member templates is rare, so this technique is currently anything but portable. In the future, that will change, but nobody knows just how far in the future that will be. Second, the mechanics of why this works are far from transparent, relying as they do on a detailed understanding of argument-matching rules for function calls, implicit type conversion functions, implicit instantiation of template functions, and the existence of member function templates.

Item 29: Reference Counting

• This is a trick worth knowing: nesting a struct in the private part of a class is a convenient way to give access to the struct to all the members of the class, but to deny access to everybody else (except, of course, friends of the class).
• A first look at the String implementation with reference counting
• The idea - that of sharing a value with other objects until we have to write on our onw copy of the value - has a long and distinguished history in Computer Science, especially in operating systems, where processes are routinely allowed to share pages until they want to modify data on their own copy of a page. The technique is common enough to have a name: copy-on-write. It's a specific example of a more general approach to efficiency, that of lazy evaluation.
  
• Reference counting is an optimization technique predicated on the assumption that objects will commonly share values \(see also Item 18\). If this assumption fails to hold, reference counting will use more memory than a more conventional implementation and it will execute more code.
  
• On the other hand, if your objects do tend to have common values, reference counting should save you both time and space. The bigger your object values and the more objects that can simultaneously share values, the more memory you'll save. The more you copy and assign values between objects, the more time you'll save. The more expensive it is to create and destroy a value, the more time you'll save there, too.
  
• In short, reference counting is most useful for improving efficiency under the following conditions:
  
  • Relatively few values are shared by relatively many objects. Such sharing typically arises through calls to assignment operators and copy constructors. The higher the objects/values ratio, the better the case for reference counting.
    
  • Object values are expensive to create or destroy, or they use lots of memory. Even when this is the case, reference counting still buys you nothing unless these values can be shared by multiple objects.
    
• Even when the conditions above are satisfied, a design employing reference counting may still be inappropriate. Some data structures (e.g., directed graphs) lead to self-referential or circular dependency structures. Such data structures have a tendency to spawn isolated collections of objects, used by no one, whose reference counts never drop to zero. That's because each object in the unused structure is pointed to by at least one other object in the same structure.
  

Item 30: Proxy classes

• Objects that stand for other objects are often called proxy objects, and the classes that give rise to proxy objects are often called proxy classes.
• n this example, Array1D is a proxy class. Its instances stand for one-dimensional arrays that, conceptually, do not exist. (The terminology for proxy objects and classes is far from universal; objects of such classes are also sometimes known as surrogates.)
• Compilers choose between const and non-const member functions by looking only at whether the object invoking a function is const. No consideration is given to the context in which a call is made
• There are only three things you can do with a proxy:
  • Create it, i.e., specify which string character it stands for.
  • Use it as the target of an assignment, in which case you are really making an assignment to the string character it stands for. When used in this way, a proxy represents an lvalue use of the string on which operator[] was invoked.
  • Use it in any other way. When used like this, a proxy represents an rvalue use of the string on which operator[] was invoked.
• In general, taking the address of a proxy yields a different type of pointer than does taking the address of a real object.
• shifting from a class that works with real objects to a class that works with proxies often changes the semantics of the class, because proxy objects usually exhibit behavior that is subtly different from that of the real objects they represent. Sometimes this makes proxies a poor choice when designing a system, but in many cases there is little need for the operations that would make the presence of proxies apparent to clients.

• class String { // reference-counted strings; public: // see Item 29 for details
  
  class CharProxy {               // proxies for string chars
  public:
      CharProxy(String& str, int index);                // creation
  
      CharProxy& operator=(const CharProxy& rhs);       // lvalue
      CharProxy& operator=(char c);                     // uses
  
      operator char() const;                            // rvalue
      // use
  private:
      String& theString;            // string this proxy pertains to
  
      int charIndex;                // char within that string
      // this proxy stands for
  };
  
  // continuation of String class
  const CharProxy operator[](int index) const;   // for const Strings
  
  CharProxy operator[](int index); // for non-const Strings
  friend class CharProxy;
  
  private: RCPtr<StringValue> value; };
  
  const String::CharProxy String::operator[](int index) const { return CharProxy(const_cast<String&>(*this), index); }
  
  String::CharProxy String::operator[](int index) { return CharProxy(*this, index); }
  
  String::CharProxy::CharProxy(String& str, int index) : theString(str), charIndex(index) {}
  
  String::CharProxy& String::CharProxy::operator=(const CharProxy& rhs) { // if the string is sharing a value with other String objects, // break off a separate copy of the value for this string only if (theString.value->isShared()) { theString.value = new StringValue(theString.value->data); }
  
  // now make the assignment: assign the value of the char
  // represented by rhs to the char represented by *this
  theString.value-&gt;data[charIndex] =
      rhs.theString.value-&gt;data[rhs.charIndex];
  
  return *this;
  }
  
  String::CharProxy& String::CharProxy::operator=(char c) { if (theString.value->isShared()) { theString.value = new StringValue(theString.value->data); }
  
  theString.value->data[charIndex] = c;
  
  return *this;
  
  }
  

Item 31: Making functions virtual with respect to more than one object

• Note the use of the unnamed namespace to contain the functions used to implement processCollision. Everything in such an unnamed namespace is private to the current translation unit (essentially the current file) - it's just like the functions were declared static at file scope. With the advent of namespaces, however, statics at file scope have been deprecated, so you should accustom yourself to using unnamed namespaces as soon as your compilers support them.

• class MyTest: {}
  namespace { // unnamed namespace, see below
  
  // primary collision-processing functions 
  void shipAsteroid(GameObject& spaceShip, GameObject& asteroid);
  
  void shipStation(GameObject& spaceShip, GameObject& spaceStation);
  
  void asteroidStation(GameObject& asteroid, GameObject& spaceStation); 
  
  // secondary collision-processing functions that just 
  // implement symmetry: swap the parameters and call a 
  // primary function 
  void asteroidShip(GameObject& asteroid, GameObject& spaceShip) {shipAsteroid(spaceShip, asteroid); }
  void stationShip(GameObject& spaceStation, GameObject& spaceShip) { shipStation(spaceShip, spaceStation); }
  void stationAsteroid(GameObject& spaceStation, GameObject& asteroid) { asteroidStation(asteroid, spaceStation); }
  
  typedef void (*HitFunctionPtr)(GameObject&, GameObject&); 
  typedef map< pair<string,string>, HitFunctionPtr > HitMap;
  
  pair<string,string> makeStringPair(const char *s1, const char *s2);
  
  HitMap * initializeCollisionMap();
  
  HitFunctionPtr lookup(const string& class1, const string& class2);
  
  } // end namespace
  
  void processCollision(GameObject& object1, GameObject& object2) { 
      HitFunctionPtr phf = lookup(typeid(object1).name(), typeid(object2).name());
      if (phf) phf(object1, object2); 
      else throw UnknownCollision(object1, object2); 
  }
  
  class CollisionMap { 
  public: 
  typedef void (*HitFunctionPtr)(GameObject&, GameObject&);
  
  void addEntry(const string& type1, const string& type2, HitFunctionPtr collisionFunction, bool symmetric = true); 
  // see below
  
  void removeEntry(const string& type1, const string& type2);
  
  HitFunctionPtr lookup(const string& type1, const string& type2);
  
  // this function returns a reference to the one and only static CollisionMap& theCollisionMap();
  
  private: // these functions are private to prevent the creation 
  CollisionMap(); 
  CollisionMap(const CollisionMap&); };
  
  class RegisterCollisionFunction { 
  public: 
  RegisterCollisionFunction( const string& type1, 
                             const string& type2, 
                             CollisionMap::HitFunctionPtr collisionFunction, 
                             bool symmetric = true) 
  { CollisionMap::theCollisionMap().addEntry(type1, type2, collisionFunction, symmetric); } 
  };
  

Item 32: Program in the future tense

Item 33: Make non-leaf classes abstract

• Still, the general rule remains: non-leaf classes should be abstract. You may need to bend the rule when working with outside libraries, but in code over which you have control, adherence to it will yield dividends in the form of increased reliability, robustness, comprehensibility, and extensibility throughout your software.

Item 34: Understand how to combine C++ and C in the same program

• there are four other things you need to consider: name mangling, initialization of statics, dynamic memory allocation, and data structure compatibility
• To suppress name mangling, use C++'s extern "C" directive
• you need to deal with the fact that in C++, lots of code can get executed before and after main. In particular, the constructors of static class objects and objects at global, namespace, and file scope are usually called before the body of main is executed. This process is known as static initialization (see Item E47).
• If you want to mix C++ and C in the same program, remember the following simple guidelines:
  • Make sure the C++ and C compilers produce compatible object files.
  • Declare functions to be used by both languages extern "C".
  • If at all possible, write main in C++.
  • Always use delete with memory from new; always use free with memory from malloc.
  • Limit what you pass between the two languages to data structures that compile under C; the C++ version of structs may contain non-virtual member functions.

Item 35: Familiarize yourself with °the language standard.

• The STL is based on three fundamental concepts: containers, iterators, and algorithms.