2017年8月22日 星期二

Gitlab email notification related to pipeline

Gitlab has email notification on various events such as pipeline failure / success, commitment etc. Unfortunately, the setting itself is confusing which make it hard to configure.


My Wish
When I tag a release, pipeline runs, after that send the pipeline results to the person I want.

You can see only a small request takes me half day to figure out where the configuration is. Here is how it can be achieve and the misleading configurations.

You may think that the notification should be well configured via "project > notifications", then you are wrong, the desired recipients will never receive the message, because that setting means ONLY AUTHOR will receive the pipeline status changes. I am quite surprise, not because of the result, but because of the lack of explanation regarding on the configurations.














So how can we achieve to send the email notification BESIDES AUTHOR? You may need to navigate to "Project > Settings > Integrations", enable "Pipelines emails", enter the desired email addresses and gotcha.

What a genius configuration settings, I really hope that software designers can design software based on "human beings" but not only for engineering mindset. It is unacceptable for a task being so complicated to accomplish.

2017年8月17日 星期四

Configuring git-lab runner in Windows 10 Home

Background
One of a routine task during my development flow is deployment, this is quite messy and time consuming, especially when the system consist of many sub-systems which may require testing and runner to run in different machines. In my case, I have developed a new feature which send email using localhost through PHP, and I have tested this feature using docker in my Windows 10 laptop. Unfortunately, Windows 10 Home does not featured with hyper-V, it is an essential element to run native Docker. I therefore make use of docker toolbox and virtual box combination as the runner of my Lenovo laptop.

Environment
- Lenovo ThinkPad X260
- Windows 10 Home
- Gitlab runner installed
- Docker toolbox installed
- Virtual box installed

Problems
All the installation processes of different components performed successfully, however when it comes to some setup, problem comes by.

- When the job is running and using the runner, the following error occurs
error during connect: Get http://%2F%2F.%2Fpipe%2Fdocker_engine/v1.29/version: open //./pipe/docker_engine: The system cannot find the file specified. In the default daemon configuration on Windows, the docker client must be run elevated to connect. This error may also indicate that the docker daemon is not running.

This indicates the docker engine is not running at the host machine. I feel strange as docker toolbox has already installed and well configured. The problem comes from the config.toml, this is the configuration file to setup the runner

concurrent = 1
check_interval = 0

[[runners]]
  name = "lenovo_laptop_jackylam"
  url = "https://repo.gitlab.dev.internal.pm.com/"
  token = "b4032157c339f90523a77e5e8881a7"
  executor = "docker"
  [runners.docker]
    tls_verify = true
    host = "tcp://192.168.99.100:2376"
    tls_cert_path = "C:\\Users\\p2m\\.docker\\machine\\certs\\"
    image = "test/2938:latest"
    pull_policy = 'never'
    privileged = true
    disable_cache = false
    volumes = ["/cache"]
    shm_size = 0
  [runners.cache]

Here, executor is important, you need to told gitlab-runner to use docker as the runner, but the problem also come from this, when you say "executor: docker", you are telling the specified host machine to run docker command for the test, the problem is you are using virtual box as the intermediate to run the docker, and the "Docker Quickstart Terminal" 's environment is linked to the virtual machine, that's why you can directly run docker command through the host terminal. When gitlab-runner run the docker command, the environment does not contain docker executable!

To solve this issue, you can rely on config.toml, in [runners.docker] column, specify the host with the virtual machine's host, and also the certifications required to run the command, another important setup would be "pull_policy", setting "never" tells gitlab-runner not to pull the docker images from the docker hub, it is extremely useful when your test is not replying on the docker images situated in docker hub. After setting up the remote connection pointing to the virtual machine, the docker command can be run successfully.

- When you want to remove the docker-machine, make sure the virtual machine in virtual box is completely shutdown before removing docker-machine, otherwise error will occur during removal.

Remember if you don't specify anything, gitlab-runner will assume the host machine can be "docker-runnable", but actually the runner is located in the VM.


2016年12月10日 星期六

Troubleshooting on connecting device via bluetooth adapter

Device: Orico BT-403

Handfree: AVEI Music

When connecting each other, under Ubuntu 14.04, there is no extra driver installation, it will automatically paired up and connected, but one thing to remind.

There is a chance of successful connection but audio device is not shown in audio list, in this case, PulseAudio Volume Control is required to install and the following command is required to refresh the device list

 sudo -i pactl load-module module-bluetooth-discover

Reference
http://askubuntu.com/questions/481951/bluetooth-headset-connects-but-not-showing-in-sound-settings/481958#481958?newreg=55ef497722e743faa525160cecc3cc4f

2016年9月30日 星期五

Changing Unity 3D to Unity 2D

Changing default behavior to unity 2D when login needs to follow the steps below


  1. Install the deb package sudo apt-get install unity-2d
  2. rm /usr/share/xsessions/ubuntu.desktop (You can rename it if you want unity 3d)
  3. /bin/sed -i "/Session*/c Session=gnome-fallback-compiz" /var/cache/lightdm/dmrc/*.dmrc (Change the default session to gnome-fallback-compiz)
  4. Restart your machine and the default UI of Ubuntu will become Unity 2D
References
  1. http://askubuntu.com/questions/62833/how-do-i-change-the-default-session-for-when-using-auto-logins
  2. https://wiki.ubuntu.com/LightDM
  3. https://ubuntuforums.org/showthread.php?t=2245261
  4. http://askubuntu.com/questions/456766/how-to-set-default-session-in-ubuntu-14-04-lts
Environment
  • Ubuntu 14.04 

2016年9月16日 星期五

Wordpress Mail Function - Points to Note

When dealing with smtp sending email, we may encounter situations that despite all the settings are correct, email still cannot be sent successfully, this is due to the google account setting, if you also encounter similar issues, go to Goggle > My Account > turn "Allow less secure apps" to "ON".









2016年5月20日 星期五

Installing VMWare Player in Ubuntu 32 bit

Struggled in VMWare player installed in Ubuntu 14.04 32 bit, the below solution helps.

Basically, the error message is related to vmnet module, like the following

Using 2.6.x kernel build system.
make: Entering directory `/tmp/modconfig-c7FgGQ/vmmon-only'
/usr/bin/make -C /lib/modules/3.16.0-71-generic/build/include/.. SUBDIRS=$PWD SRCROOT=$PWD/. \
 MODULEBUILDDIR= modules
make[1]: Entering directory `/usr/src/linux-headers-3.16.0-71-generic'
  CC [M]  /tmp/modconfig-c7FgGQ/vmmon-only/linux/driver.o
  CC [M]  /tmp/modconfig-c7FgGQ/vmmon-only/linux/driverLog.o
  CC [M]  /tmp/modconfig-c7FgGQ/vmmon-only/linux/hostif.o
  CC [M]  /tmp/modconfig-c7FgGQ/vmmon-only/common/apic.o
  CC [M]  /tmp/modconfig-c7FgGQ/vmmon-only/common/comport.o
In file included from /tmp/modconfig-c7FgGQ/vmmon-only/linux/driverLog.h:33:0,
                 from /tmp/modconfig-c7FgGQ/vmmon-only/linux/driverLog.c:31:
/tmp/modconfig-c7FgGQ/vmmon-only/./include/vm_assert.h:259:0: warning: "DEPRECATED" redefined [enabled by default]
    #define DEPRECATED(_fix) do {} while (0)
 ^
In file included from include/linux/kernel.h:13:0,
                 from /tmp/modconfig-c7FgGQ/vmmon-only/./include/compat_kernel.h:23,
                 from /tmp/modconfig-c7FgGQ/vmmon-only/linux/driverLog.c:27:
include/linux/printk.h:106:0: note: this is the location of the previous definition
 #define DEPRECATED "[Deprecated]: "
 ^
  CC [M]  /tmp/modconfig-c7FgGQ/vmmon-only/common/cpuid.o
In file included from /tmp/modconfig-c7FgGQ/vmmon-only/./include/vmware.h:39:0,
                 from /tmp/modconfig-c7FgGQ/vmmon-only/linux/driver.c:46:
/tmp/modconfig-c7FgGQ/vmmon-only/./include/vm_assert.h:259:0: warning: "DEPRECATED" redefined [enabled by default]
    #define DEPRECATED(_fix) do {} while (0)
 ^
In file included from include/linux/kernel.h:13:0,
                 from ./arch/x86/include/asm/percpu.h:44,
                 from ./arch/x86/include/asm/preempt.h:5,
                 from include/linux/preempt.h:18,
                 from include/linux/spinlock.h:50,
                 from include/linux/wait.h:8,
                 from include/linux/fs.h:6,
                 from include/linux/highmem.h:4,
                 from /tmp/modconfig-c7FgGQ/vmmon-only/linux/driver.c:25:
include/linux/printk.h:106:0: note: this is the location of the previous definition
 #define DEPRECATED "[Deprecated]: "
 ^
In file included from /tmp/modconfig-c7FgGQ/vmmon-only/./include/vmware.h:39:0,
                 from /tmp/modconfig-c7FgGQ/vmmon-only/linux/hostif.c:82:
/tmp/modconfig-c7FgGQ/vmmon-only/./include/vm_assert.h:259:0: warning: "DEPRECATED" redefined [enabled by default]
    #define DEPRECATED(_fix) do {} while (0)
 ^
In file included from include/linux/kernel.h:13:0,
                 from include/asm-generic/bug.h:13,
                 from ./arch/x86/include/asm/bug.h:35,
                 from include/linux/bug.h:4,
                 from include/linux/mmdebug.h:4,
                 from include/linux/mm.h:8,
                 from /tmp/modconfig-c7FgGQ/vmmon-only/./include/compat_page.h:23,
                 from /tmp/modconfig-c7FgGQ/vmmon-only/linux/hostif.c:32:
include/linux/printk.h:106:0: note: this is the location of the previous definition
 #define DEPRECATED "[Deprecated]: "
 ^
  CC [M]  /tmp/modconfig-c7FgGQ/vmmon-only/common/hashFunc.o
/tmp/modconfig-c7FgGQ/vmmon-only/linux/driver.c:1342:1: warning: always_inline function might not be inlinable [-Wattributes]
 LinuxDriverSyncReadTSCs(uint64 *delta) // OUT: TSC max - TSC min
 ^
  CC [M]  /tmp/modconfig-c7FgGQ/vmmon-only/common/memtrack.o
  CC [M]  /tmp/modconfig-c7FgGQ/vmmon-only/common/phystrack.o
  CC [M]  /tmp/modconfig-c7FgGQ/vmmon-only/common/task.o
  CC [M]  /tmp/modconfig-c7FgGQ/vmmon-only/common/vmx86.o
In file included from /tmp/modconfig-c7FgGQ/vmmon-only/./include/vmware.h:39:0,
                 from /tmp/modconfig-c7FgGQ/vmmon-only/common/vmx86.c:40:
/tmp/modconfig-c7FgGQ/vmmon-only/./include/vm_assert.h:259:0: warning: "DEPRECATED" redefined [enabled by default]
    #define DEPRECATED(_fix) do {} while (0)
 ^
In file included from include/linux/kernel.h:13:0,
                 from include/linux/sched.h:17,
                 from /tmp/modconfig-c7FgGQ/vmmon-only/common/vmx86.c:31:
include/linux/printk.h:106:0: note: this is the location of the previous definition
 #define DEPRECATED "[Deprecated]: "
 ^
  CC [M]  /tmp/modconfig-c7FgGQ/vmmon-only/vmcore/moduleloop.o
In file included from /tmp/modconfig-c7FgGQ/vmmon-only/./include/vmware.h:39:0,
                 from /tmp/modconfig-c7FgGQ/vmmon-only/linux/driver.c:46:
/tmp/modconfig-c7FgGQ/vmmon-only/./include/vm_assert.h:259:0: warning: "DEPRECATED" redefined [enabled by default]
    #define DEPRECATED(_fix) do {} while (0)
 ^
In file included from include/linux/kernel.h:13:0,
                 from ./arch/x86/include/asm/percpu.h:44,
                 from ./arch/x86/include/asm/preempt.h:5,
                 from include/linux/preempt.h:18,
                 from include/linux/spinlock.h:50,
                 from include/linux/wait.h:8,
                 from include/linux/fs.h:6,
                 from include/linux/highmem.h:4,
                 from /tmp/modconfig-c7FgGQ/vmmon-only/linux/driver.c:25:
include/linux/printk.h:106:0: note: this is the location of the previous definition
 #define DEPRECATED "[Deprecated]: "
 ^
In file included from /tmp/modconfig-c7FgGQ/vmmon-only/./include/vmware.h:39:0,
                 from /tmp/modconfig-c7FgGQ/vmmon-only/vmcore/moduleloop.c:33:
/tmp/modconfig-c7FgGQ/vmmon-only/./include/vm_assert.h:259:0: warning: "DEPRECATED" redefined [enabled by default]
    #define DEPRECATED(_fix) do {} while (0)
 ^
In file included from include/linux/kernel.h:13:0,
                 from /tmp/modconfig-c7FgGQ/vmmon-only/vmcore/moduleloop.c:30:
include/linux/printk.h:106:0: note: this is the location of the previous definition
 #define DEPRECATED "[Deprecated]: "
 ^
  LD [M]  /tmp/modconfig-c7FgGQ/vmmon-only/vmmon.o
  Building modules, stage 2.
  MODPOST 1 modules
  CC      /tmp/modconfig-c7FgGQ/vmmon-only/vmmon.mod.o
  LD [M]  /tmp/modconfig-c7FgGQ/vmmon-only/vmmon.ko
make[1]: Leaving directory `/usr/src/linux-headers-3.16.0-71-generic'
/usr/bin/make -C $PWD SRCROOT=$PWD/. \
 MODULEBUILDDIR= postbuild
make[1]: Entering directory `/tmp/modconfig-c7FgGQ/vmmon-only'
make[1]: `postbuild' is up to date.
make[1]: Leaving directory `/tmp/modconfig-c7FgGQ/vmmon-only'
cp -f vmmon.ko ./../vmmon.o
make: Leaving directory `/tmp/modconfig-c7FgGQ/vmmon-only'
Using 2.6.x kernel build system.
make: Entering directory `/tmp/modconfig-c7FgGQ/vmnet-only'
/usr/bin/make -C /lib/modules/3.16.0-71-generic/build/include/.. SUBDIRS=$PWD SRCROOT=$PWD/. \
 MODULEBUILDDIR= modules
make[1]: Entering directory `/usr/src/linux-headers-3.16.0-71-generic'
  CC [M]  /tmp/modconfig-c7FgGQ/vmnet-only/driver.o
  CC [M]  /tmp/modconfig-c7FgGQ/vmnet-only/hub.o
  CC [M]  /tmp/modconfig-c7FgGQ/vmnet-only/userif.o
  CC [M]  /tmp/modconfig-c7FgGQ/vmnet-only/netif.o
In file included from /tmp/modconfig-c7FgGQ/vmnet-only/monitorAction_exported.h:31:0,
                 from /tmp/modconfig-c7FgGQ/vmnet-only/vnet.h:29,
                 from /tmp/modconfig-c7FgGQ/vmnet-only/vnetInt.h:24,
                 from /tmp/modconfig-c7FgGQ/vmnet-only/hub.c:43:
/tmp/modconfig-c7FgGQ/vmnet-only/vm_assert.h:259:0: warning: "DEPRECATED" redefined [enabled by default]
    #define DEPRECATED(_fix) do {} while (0)
 ^
In file included from include/linux/kernel.h:13:0,
                 from /tmp/modconfig-c7FgGQ/vmnet-only/hub.c:23:
include/linux/printk.h:106:0: note: this is the location of the previous definition
 #define DEPRECATED "[Deprecated]: "
 ^
In file included from /tmp/modconfig-c7FgGQ/vmnet-only/monitorAction_exported.h:31:0,
                 from /tmp/modconfig-c7FgGQ/vmnet-only/vnet.h:29,
                 from /tmp/modconfig-c7FgGQ/vmnet-only/vnetInt.h:24,
                 from /tmp/modconfig-c7FgGQ/vmnet-only/userif.c:45:
/tmp/modconfig-c7FgGQ/vmnet-only/vm_assert.h:259:0: warning: "DEPRECATED" redefined [enabled by default]
    #define DEPRECATED(_fix) do {} while (0)
 ^
In file included from include/linux/kernel.h:13:0,
                 from ./arch/x86/include/asm/percpu.h:44,
                 from ./arch/x86/include/asm/preempt.h:5,
                 from include/linux/preempt.h:18,
                 from include/linux/spinlock.h:50,
                 from include/linux/wait.h:8,
                 from include/linux/fs.h:6,
                 from include/linux/highmem.h:4,
                 from /tmp/modconfig-c7FgGQ/vmnet-only/userif.c:26:
include/linux/printk.h:106:0: note: this is the location of the previous definition
 #define DEPRECATED "[Deprecated]: "
 ^
/tmp/modconfig-c7FgGQ/vmnet-only/hub.c: In function ‘VNetHubFindHubByID’:
/tmp/modconfig-c7FgGQ/vmnet-only/hub.c:132:49: warning: argument to ‘sizeof’ in ‘__builtin_memcmp’ call is the same expression as the first source; did you mean to provide an explicit length? [-Wsizeof-pointer-memaccess]
         memcmp(idNum, currHub->id.pvnID, sizeof idNum))) {
                                                 ^
In file included from ./arch/x86/include/asm/string.h:2:0,
                 from include/linux/string.h:17,
                 from ./arch/x86/include/asm/page_32.h:34,
                 from ./arch/x86/include/asm/page.h:13,
                 from ./arch/x86/include/asm/thread_info.h:11,
                 from include/linux/thread_info.h:54,
                 from ./arch/x86/include/asm/preempt.h:6,
                 from include/linux/preempt.h:18,
                 from include/linux/spinlock.h:50,
                 from include/linux/seqlock.h:35,
                 from include/linux/time.h:5,
                 from include/uapi/linux/timex.h:56,
                 from include/linux/timex.h:56,
                 from include/linux/sched.h:19,
                 from /tmp/modconfig-c7FgGQ/vmnet-only/hub.c:25:
/tmp/modconfig-c7FgGQ/vmnet-only/hub.c: In function ‘VNetHubAlloc’:
/tmp/modconfig-c7FgGQ/vmnet-only/hub.c:315:36: warning: argument to ‘sizeof’ in ‘__builtin_memcpy’ call is the same pointer type ‘uint8 *’ as the destination; expected ‘uint8’ or an explicit length [-Wsizeof-pointer-memaccess]
   memcpy(hub->id.pvnID, id, sizeof id);
                                    ^
./arch/x86/include/asm/string_32.h:182:48: note: in definition of macro ‘memcpy’
 #define memcpy(t, f, n) __builtin_memcpy(t, f, n)
                                                ^
In file included from /tmp/modconfig-c7FgGQ/vmnet-only/monitorAction_exported.h:31:0,
                 from /tmp/modconfig-c7FgGQ/vmnet-only/vnet.h:29,
                 from /tmp/modconfig-c7FgGQ/vmnet-only/vnetInt.h:24,
                 from /tmp/modconfig-c7FgGQ/vmnet-only/netif.c:42:
/tmp/modconfig-c7FgGQ/vmnet-only/vm_assert.h:259:0: warning: "DEPRECATED" redefined [enabled by default]
    #define DEPRECATED(_fix) do {} while (0)
 ^
In file included from include/linux/kernel.h:13:0,
                 from /tmp/modconfig-c7FgGQ/vmnet-only/netif.c:23:
include/linux/printk.h:106:0: note: this is the location of the previous definition
 #define DEPRECATED "[Deprecated]: "
 ^
/tmp/modconfig-c7FgGQ/vmnet-only/netif.c: In function ‘VNetNetIf_Create’:
/tmp/modconfig-c7FgGQ/vmnet-only/netif.c:151:79: error: macro "alloc_netdev" passed 4 arguments, but takes just 3
    dev = alloc_netdev(sizeof *netIf, deviceName, NET_NAME_USER, VNetNetIfSetup);
                                                                               ^
/tmp/modconfig-c7FgGQ/vmnet-only/netif.c:151:10: error: ‘alloc_netdev’ undeclared (first use in this function)
    dev = alloc_netdev(sizeof *netIf, deviceName, NET_NAME_USER, VNetNetIfSetup);
          ^
/tmp/modconfig-c7FgGQ/vmnet-only/netif.c:151:10: note: each undeclared identifier is reported only once for each function it appears in
/tmp/modconfig-c7FgGQ/vmnet-only/netif.c: At top level:
/tmp/modconfig-c7FgGQ/vmnet-only/netif.c:85:1: warning: ‘VNetNetIfSetup’ defined but not used [-Wunused-function]
 VNetNetIfSetup(struct net_device *dev)  // IN:
 ^
make[2]: *** [/tmp/modconfig-c7FgGQ/vmnet-only/netif.o] Error 1
make[2]: *** Waiting for unfinished jobs....
In file included from /tmp/modconfig-c7FgGQ/vmnet-only/monitorAction_exported.h:31:0,
                 from /tmp/modconfig-c7FgGQ/vmnet-only/vnet.h:29,
                 from /tmp/modconfig-c7FgGQ/vmnet-only/vnetInt.h:24,
                 from /tmp/modconfig-c7FgGQ/vmnet-only/driver.c:51:
/tmp/modconfig-c7FgGQ/vmnet-only/vm_assert.h:259:0: warning: "DEPRECATED" redefined [enabled by default]
    #define DEPRECATED(_fix) do {} while (0)
 ^
In file included from include/linux/kernel.h:13:0,
                 from /tmp/modconfig-c7FgGQ/vmnet-only/driver.c:23:
include/linux/printk.h:106:0: note: this is the location of the previous definition
 #define DEPRECATED "[Deprecated]: "
 ^
make[1]: *** [_module_/tmp/modconfig-c7FgGQ/vmnet-only] Error 2
make[1]: Leaving directory `/usr/src/linux-headers-3.16.0-71-generic'
make: *** [vmnet.ko] Error 2
make: Leaving directory `/tmp/modconfig-c7FgGQ/vmnet-only'
Unable to install all modules.  See log for details.

Seems that vmware is not able to compile the network module under kernel 3.16, so we need to apply this patch

http://askubuntu.com/questions/449629/error-installing-vmware-workstation-on-ubuntu/449630#449630?newreg=0be708f4a9c0411d973eb3035b176d69

 Details of the patch will be analyzed later

After patching the vmware, just start the vmware player as usual and install the module

2016年4月23日 星期六

Backup and Restore of MongoDB

In MySQL world, we can backup and restore our database pretty easy via phpmyadmin or mysqldump etc. But in mongoDB, it is even easier! Just running 2 commands in total, one for backup and one for restore, and you can seamlessly transfer your mongoDB data to the new machine.


In my case, I try to transfer the DB data from my old raspberry gen1 to my new raspberry pi 3, of course, it works on all sorts of Linux platform and Windows also.


First, to backup your DB, run

mongodump --db myDB --out destPath

This is the simplest backup method, of course, mongodump provides many options for customization, but in most situation, this would be enough. If you want to backup all the databases, do not specify --db.

Then you just copy the dumped mongoDB file to the desired location and run
mongorestore --db myDB --drop /data/db

/data/db is my designated path, you can have it yourself, after that, all the data will be restored! Easy right? Remember --drop parameter is important as changes in the database will not be restored if not specified.


Update: 19-6-2016
Remember, the restore path is always the relative path of the current directory, i.e. Say
mongorestore --drop -d db abc/

"abc" is a RELATIVE directory of the current shell

Reference:

  1. http://stackoverflow.com/questions/3884418/mongorestore-of-a-db-causing-me-trouble
  2. https://docs.mongodb.org/manual/reference/program/mongodump/#bin.mongodump
  3. https://docs.mongodb.org/manual/reference/program/mongorestore/