:: Server

Showing posts with label Server. Show all posts

2021-07-01

Remove password from pdf file using ghostscript

$ gs -q -dNOPAUSE -dBATCH -sDEVICE=pdfwrite -sPDFPassword=PASSWORD -sOutputFile=OUTPUT.pdf -f ENCRYPTED.pdf

Install GhostScript on Mac

$ brew install ghostscript

2020-09-28

Unicode normalization

NFD to NFC with convmv

 $ brew install convmv

 # Convert all files in a directory from NFD to NFC:
 $ convmv -f utf-8 -t utf-8 --nfc --notest .

 # Convert all files in a directory from NFC to NFD:
 $ convmv -f utf-8 -t utf-8 --nfd --notest .

NFD to NFC with iconv

 $ iconv -f UTF-8 -t UTF-8-MAC

NFD to NFC with rsync

 $ rsync --iconv=utf-8,utf-8-mac

2020-08-27

SSH with multiple private keys

Add privat keys

 $ ssh-add ~/.ssh/id_rsa_SERVER

Verify keys

 $ ssh-add -l

Test keys

 $ ssh -v URL_TO_HOST

Connect to server

 $ ssh HOSTNAME_OR_ALIAS

2020-04-23

ISO to USB on Mac Mojave

List devices

$ diskutil list
/dev/disk0 (internal, physical):
   #:                       TYPE NAME                    SIZE       IDENTIFIER
   0:      GUID_partition_scheme                        *251.0 GB   disk0
   1:                        EFI EFI                     314.6 MB   disk0s1
   2:                 Apple_APFS Container disk1         250.7 GB   disk0s2

/dev/disk1 (synthesized):
   #:                       TYPE NAME                    SIZE       IDENTIFIER
   0:      APFS Container Scheme -                      +250.7 GB   disk1
                                 Physical Store disk0s2
   1:                APFS Volume Macintosh HD - Data     77.5 GB    disk1s1
   2:                APFS Volume Preboot                 82.0 MB    disk1s2
   3:                APFS Volume Recovery                528.1 MB   disk1s3
   4:                APFS Volume VM                      1.1 GB     disk1s4
   5:                APFS Volume Macintosh HD            11.1 GB    disk1s5

/dev/disk2 (external, physical):
   #:                       TYPE NAME                    SIZE       IDENTIFIER
   0:     FDisk_partition_scheme                        *124.2 GB   disk2
   1:             Windows_FAT_32 CCCOMA_X64F             124.2 GB   disk2s1

Unmount USB stick

$ diskutil unmountDisk /dev/disk2
Unmount of all volumes on disk2 was successful

$ sudo dd if=kali-linux-2020.1b-installer-amd64.iso of=/dev/disk2 bs=1m
Password:
2075+1 records in
2075+1 records out
2176778240 bytes transferred in 656.146374 secs (3317519 bytes/sec)

2019-11-04

Install Mysql, Mariadb, Postgresql and Mongodb on Docker on Mac

Sort reverse by Unicode string length Raw

How variables are represented in memory

Convert PNG file to ICNS file on Mac

2019-10-24

Install unbound dns cache on Mac

2019-10-09

Clear Mac Finder and Dock icon cache, and Reset Launchpad layout

Clear Mac Finder and Dock icon cache

2019-06-26

Allow SSH access based on Country using GeoIP

Install packages

$ sudo apt install geoip-bin geoip-database

Create script

$ sudo vi /usr/local/bin/sshfilter.sh


#!/bin/bash

ALLOWED_COUNTRIES="KR US GB NZ"
ALLOWED_IP="192.168 127.0.0"

[[ "`echo $1 | grep ':'`" != "" ]] && SUFFIX="6"
if [[ $ALLOWED_IP =~ ${1:0:7} ]]; then
 logger -p authpriv.notice "SSH ALLOWED $1 LOCALDOMAIN"
 exit 0
fi

GEORESULT=`/usr/bin/geoiplookup${SUFFIX} "$1"`
COUNTRY=`echo $GEORESULT | cut -d : -f 2 | xargs | head -1`
COUNTRYCODE=`echo $COUNTRY | cut -d , -f 1 | xargs`
[[ $ALLOWED_COUNTRIES =~ $COUNTRYCODE ]] && RESPONSE="ALLOWED" || RESPONSE="DENIED"
logger -p authpriv.notice "SSH $RESPONSE $1 $COUNTRY"
[[ $RESPONSE == "ALLOWED" ]] && exit 0 || exit 1

edit /etc/hosts.allow and /etc/hosts.deny

# /etc/hosts.allow
sshd: ALL: /usr/local/bin/sshfilter.sh %a

# /etc/hosts.deny
sshd: ALL

SSH lockout 60min after 5 brute force attempts in 10min

install fail2ban

$ sudo apt update
$ sudo apt install fail2ban

edit configuration

$ sudo vi /etc/fail2ban/jail.conf

[sshd]
enabled  = true
port     = ssh
maxretry = 5    # 5 attempts not permitted
findtime = 600  # within 10 minutes
bantime  = 3600 # ban 1 hour
logpath  = %(sshd_log)s
backend  = %(sshd_backend)s

unban an ip address

$ sudo fail2ban-client set sshd unbanip IP_ADDRESS

2017-11-10

How to use different gateways by IP address on Linux and Mac

on Linux

$ sudo route add -net 192.168.0.0 netmask 255.255.0.0 gw 192.168.0.1
$ sudo route add -net 192.168.1.0 netmask 255.255.0.0 gw 192.168.0.254
$ sudo route add -net 192.168.0.0 netmask 255.255.0.0 dev eth0
$ sudo route add -net 192.168.1.0 netmask 255.255.0.0 dev eth1

$ sudo route add -net 192.168.0.0/16 gw 192.168.0.1
$ sudo route add -net 192.168.1.0/16 gw 192.168.0.254
$ sudo route add -net 192.168.0.0/16 dev eth0
$ sudo route add -net 192.168.1.0/16 dev eth1

on Mac

$ sudo route -n add -net 192.168.0.0 -netmask 255.255.0.0 192.168.0.1
$ sudo route -n add -net 192.168.1.0 -netmask 255.255.0.0 192.168.0.254

$ sudo route -n add -net 192.168.0.0/16 192.168.0.1
$ sudo route -n add -net 192.168.1.0/16 192.168.0.254

2017-01-11

Install Spark on Ubuntu

# Install Java

$ sudo apt install python-software-properties
$ sudo add-apt-repository ppa:webupd&team/java
$ sudo apt update
$ sudo apt install oracle-java8-installer

# Install Scala

$ sudo apt install scala

# Install Spark
Download prebuilt Spark from http://spark.apache.org/downloads.html

$ wget http://d3kbcqa49mib13.cloudfront.net/spark-2.1.0-bin-hadoop2.7.tgz
$ sudo mv spark-2.1.0-bin-hadoop2.7.tgz /opt
$ sudo tar xfz spark-2.1.9-bin-hadoop2.7.tgz
$ sudo ln -s spark-2.1.9-bin-hadooop2.7 spark

# Add environment variables and add path

$ vi ~/.profile

JAVA_HOME=/usr/lib/jvm/java-8-oracle
SCALA_HOME=/usr/share/scala
SPARK_HOME=/opt/spark
PYTHONPATH=$SPARK_HOME/python/lib/pyspark.zip:$SPARK_HOME/python/lib/py4j-0.10-4-src.zip
export JAVA_HOME SPARK_HOME PYTHONPATH
PATH="$HOME/bin:$HOME/.local/bin:$PATH:$SPARK_HOME/bin"

# Edit configuration(Optioinal) to reduce logs displayed

$ cd $SPARK_HOME/conf
$ sudo cp log4j.properties.template log4j.properties
$ sudo vi log4j.properties

FIND : log4j.rootCategory=INFO, console
REPLACE : log4j.rootCategory=WARN, console

# Test if Spark works

$ run-example SparkPi 10

Pi is roughly 3.140963140963141

# Launch Spark Shell(run scala language) or PySpark(run python language)

$ spark-shell
Welcome to
      ____              __
     / __/__  ___ _____/ /__
    _\ \/ _ \/ _ `/ __/  '_/
   /___/ .__/\_,_/_/ /_/\_\   version 2.1.0
      /_/
        
Using Scala version 2.11.8 (Java HotSpot(TM) 64-Bit Server VM, Java 1.8.0_111)
Type in expressions to have them evaluated.
Type :help for more information.
scala >

$ pyspark
Welcome to
      ____              __
     / __/__  ___ _____/ /__
    _\ \/ _ \/ _ `/ __/  '_/
   /__ / .__/\_,_/_/ /_/\_\   version 2.1.0
      /_/
Using Python version 2.7.12 (default, Nov 19 2016 06:48:10)
SparkSession available as 'spark'.
 > > >

$ Spark UI

http://localhost:4040

2016-06-11

rsync exclude any folder with a name in subdirectories

$ rsync -alPqtz --delete --stats -e ssh --exclude dir/ --log-file /path/to/logfile /path/to/source user@host:/path/to/desination

/DIR/ exclude DIR from root
/DIR/* include DIR but exclude its contents
DIR/ exclude DIR from anywhere

See rsync exclude filter

Postfix for sending email only from localhost

Install Postfix and configure

$ sudo apt-get update
$ sudo apt-get install mailutils

$ sudo systemctl status postfix


● postfix.service - LSB: Postfix Mail Transport Agent
   Loaded: loaded (/etc/init.d/postfix; bad; vendor preset: enabled)
  Drop-In: /run/systemd/generator/postfix.service.d
           └─50-postfix-$mail-transport-agent.conf
   Active: active (running) since {DATE}; 8min ago
     Docs: man:systemd-sysv-generator(8)
  Process: 24767 ExecStop=/etc/init.d/postfix stop (code=exited, status=0/SUCCES
  Process: 24796 ExecStart=/etc/init.d/postfix start (code=exited, status=0/SUCC
   CGroup: /system.slice/postfix.service
           ├─{PS_NUM1} /usr/lib/postfix/sbin/master
           ├─{PS_NUM2} pickup -l -t unix -u -c\
           └─{PS_NUM3} qmgr -l -t unix -u

Test

$ echo "TEST BODY" | mail -s "TEST SUBJECT" account@email.server.com

Error
```
Check /var/log/mail.err
```

2014-10-14

mac os, port forward

sudo ipfw add 100 fwd 127.0.0.1,10080 tcp from any to me 80
sudo ipfw add 101 fwd 127.0.0.1,10443 tcp from any to me 443

2013-09-19

How to unlock encrypted file system on Linux automatically

create a keyfile with random content

$ sudo su -
$ dd if=/dev/random of=/root/keyfile bs=1024 count=4

Add the file to LUKS

$ cryptsetyp luksAddKey /dev/sdb1 /root/keyfile

Edit /etc/crypttab

$ sdb1_crypt /dev/sdb1 /root/keyfile luks

Edit /etc/fstab

/dev/mapper/sdb1_crypt /media/Repository ext4 defaults 0 2

Reboot or Remount

2013-08-27

Ternary operator(?:) in bash

Ternary operator is a short form of if/then/else

case $a in
    1) b=$c ;;
    *) b=$d ;;
esac

[[ $a=1 ]] && b=$c || b=$d
# At this case, 3rd part can be executed in case 2nd part fails

[[ $a=1 ]] && { b=$c; true; } || b=$d
# to avoid executing 3rd part by accident when 2nd part fails
## semicolon(;) is needed in {}

2013-08-23

Count number of visitors(IP addresses) with IIS log files

* Instruction

Mount IIS log directory into Linux box

$ sudo mount -t cifs -o ro,username=USER,password=PASSWD //HOST_IP/SHARE /MOUNT_POINT

Run

#!/bin/bash

site_num=(1 2 7 8)
site_addr=("AAA.com" "BBB.com" "CCC.com" "DDD.com")
echo "SITE_NAME DATE WEEK TRAFFIC DB_CONN VISITOR"
for ix in ${!site_num[*]}
do
  d=${site_num[$ix]}
  for f in $(ls W3SVC$d)
  do
    if [[ ${f:4:6} -ge $(date -d 'last monday' +"%y%m%d") && ${f:4:6} -lt $(date +"%y%m%d") ]]
    then
      visitors=$(cat W3SVC$d/$f | grep ^[0-9] | awk '{if(!match($9, /^192.168./) && !match($9, /^10.10./)) print $9}' | sort | uniq | wc -l)
      sendbytes=$(cat W3SVC$d/$f | grep ^[0-9] | awk '{print $16}' | awk '{total=total+$1}END{printf "%.f", total}')
      dbconns=$(cat W3SVC$d/$f | awk '{if(match($5,/^\/$/) || match($5,/asp/)) print $9}' | wc -l)
      weeknum=$(date -dyesterday +%V)
      [ $(date -d yesterday +%u) -eq 1 ] && {mon=$(date -d yesterday +%m.%d); true} || mon=$(date -d last-monday +%m.%d)
      [ $(date -d yesterday +%u) -eq 7 ] && {sun=$(date -d yesterday +%m.%d); true} || sun=$(date -d sunday +%m.%d)
      printf "%s 20%s-%s-%s %s(%s-%s) %s %s %s\n" ${site_addr[$ix]} ${f:4:2} ${f:6:2} ${f:8:2} $weeknum %mon %sun $sendbytes $dbconns $visitors
    fi
  done
done

* Sample result

SITE_NAME DATE WEEK TRAFFIC DB_CONN VISITOR
AAA.com 2013-08-25 34(08.19-08.25) 12329500782 54908 7335
BBB.com 2013-08-25 34(08.19-08.25) 13074318914 66700 6442
CCC.com 2013-08-25 34(08.19-08.25) 23734108053 140609 12877
DDD.com 2013-08-25 34(08.19-08.25) 8913154535 58417 5172