IoC : Ulises Fasoli's blog: bash

Showing posts with label bash. Show all posts

OSX show used ports or listening applications with their PID

On OSX you can display applications listening on a given port using the lsof the commands described below will show listening applications as well as the PID for each application

lsof -nP -i4TCP | grep LISTEN


lsof -nP -i4TCP:[PORT] | grep LISTEN

Where [PORT] is the given port you want to check for


lsof -nP -i4TCP:80 | grep LISTEN

Your output should look something similar to this :


idea      28856 ufasoli   15u  IPv4 0xdc1e11a68e7f918b      0t0  TCP 127.0.0.1:17434 (LISTEN)
idea      28856 ufasoli   52u  IPv4 0xdc1e11a690bab54b      0t0  TCP 127.0.0.1:6942 (LISTEN)
idea      28856 ufasoli  748u  IPv4 0xdc1e11a690b9dccb      0t0  TCP 127.0.0.1:63344 (LISTEN)
idea      28856 ufasoli 1076u  IPv4 0xdc1e11a68e7fa54b      0t0  TCP 127.0.0.1:54243 (LISTEN)
idea      28856 ufasoli 1082u  IPv4 0xdc1e11a67124254b      0t0  TCP 127.0.0.1:54266 (LISTEN)
rapportd  28871 ufasoli    5u  IPv4 0xdc1e11a665f15b6b      0t0  TCP *:51387 (LISTEN)
node      33709 ufasoli   28u  IPv4 0xdc1e11a66304354b      0t0  TCP 127.0.0.1:4200 (LISTEN)
java      34245 ufasoli  168u  IPv6 0xdc1e11a66dd5f11b      0t0  TCP *:52439 (LISTEN)
java      34245 ufasoli  182u  IPv6 0xdc1e11a6803af4db      0t0  TCP *:33389 (LISTEN)
java      34245 ufasoli  183u  IPv6 0xdc1e11a6803afafb      0t0  TCP *:8080 (LISTEN)

Preventing bash commands to be displayed in the history or deleting a specific command from history

It's sometimes useful when using the linux command line to prevent things to go into the bash history; for example if you're using curl to download a file while providing your username password.

This can be easily done in linux by configuring the HISTCONTROL variable in either your ~/.bashrc (which I recommend in order to have it permanently in your bash sessions) or by just changing the value for your current session.

Either way either set the variable as follows on your ~/.bashrc or on your current session

HISTCONTROL=ignorespace

If you took the ~/.bashrc approach don't forget to source it to have i

Once this is done commands that you start with a white space will be ignored :

[root@myserver] curl -o strx25.tar.bz2 --user user:password http://www.openss7.org/repos/tarballs/strx25-0.9.2.1.tar.bz2

note the whitespace before the curl command

Removing a specific line from history

Now if you have to remove a specific line from history this is done in 2 steps

Display commands in history :

[root@myserver]history
  832  docker images
  833  docker rmi 0d22948b5794
  834  docker images
  835  docker rmi 1a31905bb4e4
  836  docker build -t dse/docker .
  837  docker images
  838  docker rmi 47a8395f9714
  839  docker build -t dse/docker .
  840  docker images
  841  docker rmi fe7a4702bf61
  842  docker ps
  843  docker ps -a
  844  docker rm e3f7c83390ee
  845  docker rmi fe7a4702bf61
  846  docker build -t dse/docker .
  847  docker images
  848  docker  rmi a386794cb70c

Once you identified the culprit; delete it with the following command (for example here the line 845):

history -d 845

REST load testing with siege

Siege is a pretty nifty linux utility for HTTP/HTTPS load testing it can be downloaded here : siege homepage
Siege allows you to write a list of URLs in a file (one URL per line) the utility will then parse the file and execute the load tests according to your configuration file .siegerc or the command's arguments.

The only thing I do not like about siege is the fact that I would like to have stats per URL; when siege parses the url file it will write a log file with global statics for the siege session

Something like so :

2012-09-21 12:03:52,   1813,     121.43,          19,       0.49,       14.93,        0.16,        7.38,    1813,       0

which to my taste is not very clear

While executing the tests siege prints out statistics that are not found in the siege log file (which in my case are more useful) but if you have multiple URLs you don't know for which URL the printed stats are.

So I decided to handle the situation differently using awk and a bash script

Note that I'm no Linux expert so the script could probably be better written but here is how I broke it down :

I have 3 files

The urls.txt file (a txt file containing the URLs I want to test :

#URL1
http://myURL1/param1/param2.json
http://myURL1/param3/param2.json
#URL2
http://myURL2/param3/param2.json
http://myURL2/param3/param1.json

The AWK script file

!/#/{

   print "\n"
   print "**********************************************************"
   print "Testing URL : "$1
   print "**********************************************************"
   system ("echo Testing URL : "$1 " >> " SIEGE_OUTPUT " 2>&1")
   system ("siege " $1 "-v -b -r"REPS" -c"CONC" --mark="$1 " --log="SIEGE_LOG_FILE ">> "SIEGE_OUTPUT " 2>&1 ")
   print "\n"

}

The bash script containing all the configuration and calling the awk script

#! /bin/sh
#LOG_FILE=./siege-log
URL_FILE=./urls.txt # file containing the URLs to handle
USER=$(whoami)
NOW=$(date +"%d-%m-%Y_%H-%M-%S")
PWD=$(pwd)
SIEGE_LOG_FILE=$PWD"/log_"$NOW".log"  #siege log's file
SIEGE_OUTPUT=$PWD"/output_"$NOW".log" #where siege's output will be redirected
CONC=50 #number of concurrent users
REPS=1  # number of repetitions
echo "loading URLS from the file : "$URL_FILE
echo "Writing siege log into the file :" $SIEGE_LOG_FILE

SET -- $CONC

awk -f siege-benchmark.awk -v CONC=$CONC REPS=$REPS SIEGE_LOG_FILE=$SIEGE_LOG_FILE SIEGE_OUTPUT=$SIEGE_OUTPUT $URL_FILE

Now according to how your siegerc file is configured output can be a bit different I have the verbose mod off and benchmark mode on. Below is an excerpt from my log file

******************************************
Testing URL : http://myURL1/param1/param2.json
******************************************
** SIEGE 2.72
** Preparing 50 concurrent users for battle.
The server is now under siege...


Transactions:                     50 hits
Availability:                 100.00 %
Elapsed time:                   0.08 secs
Data transferred:               0.00 MB
Response time:                  0.04 secs
Transaction rate:             625.00 trans/sec
Throughput:                     0.03 MB/sec
Concurrency:                   24.62
Successful transactions:          50
Failed transactions:               0
Longest transaction:            0.07
Shortest transaction:           0.01

Bash - create subfolder in each folder

I recently had to add a subfolder to each folder inside a directory in a directory that had more than 100 folders it was a bit tedious to copy paste the folder inside each one of them.Thankfully this can easily done with a bash command :

cd /opt/myfolder
for f in *; do mkdir "$f/myFolder"; done