Home » Posts tagged 'Compression'

Tag Archives: Compression

ZRAM vs ZSWAP vs ZCACHE


Another interesting to “oprex” :

Summary of what to use when:

  1. ZRAM if you have no HDD/SSD swap partition.
  2. ZSWAP if you do have a HDD/SSD swap partition.
  3. ZCACHE: It does what ZSWAP does and ALSO compresses and speeds the filesystem page cache. (It is internally much more complicated and is not in the mainline kernel as it is still under development).

Summary of their implementations:

  1. ZRAM is a compressed RAM based swap device
  2. ZSWAP is a compressed Cache if you already have a swap.
  3. ZCache is a backend for a special type of Virtual RAM thingy (Transcendent memory) that can be used to cache filesystem pages or swap data.

Details:

  • ZRAM: Makes a swap device in the RAM. Pages sent here are compressed as they are stored. It has a higher priority than other swap devices: pages that are swapped out are preferentially sent to the zram device till it is full, only then are any other swap devices used.
    • Benefits: Independent of other (physical) swap devices. It can be used when there is no swap partition to expand the available memory.
    • Disadvantages: If other swap devices (HDD/SSD) are present they are not used optimally. As the zram device is an independent swap device, once it is full, any new pages that need to be swapped out are sent to next swap device directly, hence:
      1. There is a real chance of LRU (least recently used) inversion: It will be the most recently swapped data that goes to the slow disk, while inactive pages that were swapped out long ago will remain in the fast ZRAM
      2. The data sent to and read from the disk will consume a lot of bandwidth as it is uncompressed.
        • Status: Merged into the mainline kernel 3.14. Once enabled on a system, it requires some userspace configuration to set up the swap devices and use them.
  • ZSWAP: The frontswap system hooks attempts to swap out pages and uses zswap as write-back-cache for a HDD/SSD swap device: An attempt is made to compress the page and if it contains poorly compressible data it is directly written two the disk. If the data is compressed, it is stored in the pool of zswap memory. If pages are swapped out of memory when the total compressed pages in RAM exceeds a certain size, the Least Recently Used (LRU) compressed page is written to the disk as it is unlikely to be required soon.
    • Benefits: Very efficient use RAM and disk based swap. Minimizes Disk I/O by both reducing the number of writes and reads required (data is compressed and held in RAM) and by reducing the bandwidth of these I/O operaions as the data is in a compressed form.
    • Limitations: It is an enhancement of disk based swap systems and hence depends on a swap partition on the hard disk.
    • Status: Merged into the 3.11 mainline linux kernel.
  • ZCache: It is a backend for the Transcendent memory system. Transcendent memory provides a RAM-like memory that can only be accessed a page at a time by using put and get calls. This is unlike normal memory that can be accessed a byte at a time. The frontswap and cleancache systems hook attempts to swap and reclaim file-system page caches respectively and send them to the transcendent memory backends. When zcache is used as a backend, the data is compressed and stored in the RAM. When it fills up, compressed pages are evicted to the swap. (an alternate backend is RAMster which shares a pool of RAM across networked computers). Using only the frontswap frontend with the zcache backend works just like zswap. (In fact zswap is a simplified subset of zcache)
    • Benefits Provides compressed caching both for swap and for filesystem caches.
    • Status: Still not mainlined as it is very complicated and is being worked on.

The best resources I found were:

Compressing RAM with zRam


A successor to compcache, zram, has been already integrated in the Linux kernel for a while now. This means that no additional compilation nor tweaking is required to benefit from compressing memory on the fly and massively reduced swapping.

As with compache, I wanted to nicely integrate the solution into the Ubuntu Upstart deamon – hence this short article. After a couple of minutes of playing the configuration was ready.

Create file zramswap.conf in /etc/init and put the following content in it.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
description "Initializes zram swaping"
 
start on runlevel [2345]
stop on runlevel [!2345]
 
pre-start script
 
# load dependency modules
modprobe zram num_devices=2
 
# initialize the devices
echo 1073741824 > /sys/block/zram0/disksize
echo 1073741824 > /sys/block/zram1/disksize
 
# Creating swap filesystems
mkswap /dev/zram0
mkswap /dev/zram1
 
# Switch the swaps on
swapon -p 5 /dev/zram0
swapon -p 5 /dev/zram1
end script
 
post-stop script
 
# Switching off swap
swapoff /dev/zram0
swapoff /dev/zram1
 
rmmod zram
end script

Now you can start the service with sudo start zramswap (it will be automatically started on after the reboot as well).
You will benefit from 2x1GB swap files, which will be compressed and stored in the RAM. Tested on Ubuntu 13.10.

Oracle 11g index key compression for SAP R/3 4.7 almost completed


Setelah akhir bulan April 2012 lalu gw tuntasin pekerjaan untuk upgrade database Oracle dari 10g (10.2.0.4) menjadi 11g (11.2.0.2), maka sejak akhir bulan April dimulailah pekerjaan untuk melakukan kompresi. Index key compression menjadi pekerjaan yang pertama dilakukan.

Dimulai dari beberapa index yang kecil lalu menjangkau index yang lebih besar. Dan sampai hari kemaren (22 Mei 2012) hampir semua index (yang major dan digunakan) sudah terkompress. Masih tersisa 2 index terbesar (ACCTCR~0 dan VBOX~0) yang belum terkompress, index-index tabel yg tidak boleh dikompres seperti index milik tabel CDHDR, CDCLS, VBDATA, dll.

Proses index compression ini sendiri menggunakan tools dari SAP, BRTOOLS 7.20 patch 13 (patch terakhir pada saat tulisan ini dibuat adalah patch 23) dan dilakukan secara online. Hasil dari index key compression sendiri memberikan freespace yang cukup besar, yaitu lebih dari 1,6 TB. Dari sisi performance sendiri terlihat peningkatan performance query yang sangat signifikan karena extent dan block menjadi lebih kecil, penggunaan data buffer menjadi lebih efisien dan bahkan tidak ada peningkatan konsumsi memory dan CPU.

Secara umum, gw sangat satisfied dengan performance dari index key compression di Oracle 11g ini

OLTP Compression Oracle 11g mantap !!


Kemaren mencoba lagi compression untuk jenis OLTP compression di Oracle 11g & SAP R/3 4.7. Table yang dicompress adalah MSEG dengan size 126 GB. Proses compression dengan online reorganization dan parallel table 4 memakan waktu sekitar 3,5 jam (no user login).

Hasilnya table MSEG saat ini hanya berukuran 15 GB. Jadi size berkurang sekitar (126 – 15) 111 GB. Woww…

Kayaknya makin yakin aja neeh untuk segera diterapkan di sistem DEV, QA, dan PRD.

Upgrade Oracle 10.2.0.4 to Oracle 11.2.0.2 succeeded


Proses upgrade Oracle 10.2.0.4 ke Oracle 11.2.0.2 telah berhasil. Secara database sudah menunjukkan versi 11.2.0.2. Sekarang sedang dalam proses untuk download Oracle Client 11.2.0.2 karena R3trans mensyaratkan untuk versi client 11.2 untuk memastikan bahwa SAP bisa terkoneksi ke database.

Environment untuk upgrade :

OS        : AIX 5.3 TL 9

Dari aplikasi TNSPING sudah bisa dipastikan bahwa LISTENER bisa dipanggil .

Sedangkan dari command R3trans -x masih belum bisa. Masih ada error :

emaprd2:[sid]adm 1> R3trans -dThis is R3trans version 6.13 (release 640 – 19.01.11 – 17:51:00).2EETW169 no connect possible: “DBMS = ORACLE                           — dbs_ora_tnsname = “[SID]'”R3trans finished (0012).

Hal ini karena Oracle client yang digunakan masih belum sesuai. Masih versi 10.2.0.4.

Gak sabar untuk nyobain OLTP compression-nya.