ITX8080: erinevus redaktsioonide vahel

Allikas: Lambda
Mine navigeerimisribale Mine otsikasti
Anttix (arutelu | kaastöö)
Resümee puudub
Anttix (arutelu | kaastöö)
 
(ei näidata sama kasutaja 25 vahepealset redaktsiooni)
2. rida: 2. rida:
== Schedule ==
== Schedule ==


Lecture TH 12:00 - 13:30
Lecture TH 17:45 - 21:15 (odd weeks)


Practice FR 14:00 - 17:30
Practice TH 17:45 - 21:15 (even weeks)


== Communication ==
== Communication ==


Discussinons will be held at: itx8080 atatat googlegroups.com
Discussinons will be held at course forum in Moodle.


Send e-mail to hannesa at-no-spam-please gmail.com for an invitation.
https://moodle.e-ope.ee/course/view.php?id=4319
 
All scenario proposals must be posted to their own respective forums.
 
Please register your account today at http://moodle.e-ope.ee


The lecturers can be reached via the course coordinator:  
The lecturers can be reached via the course coordinator:  
16. rida: 20. rida:
antti.andreimann only-people-please eesti.ee
antti.andreimann only-people-please eesti.ee


== Points awarded so far ==
= Grading =


Arnis : 3 points - ab testing solution, best countermeasure proposed for scenario 4, best snort rules proposed for scenario 6
Best of the bunch will score points.


Igor : 3 points - wordpress virtualization solution, lab 3 report, IDS testing solution
Points will be awareded for the:


Kaarel : 2 points - lab 1 report and conclusions, lab 3 solution
* Best simulation proposal
* Best result obtained in the lab environment
* Best lab report


Märt : 3 points - wordpress testing procedure, best wordpress optimization (wp-cache + php-apc), best lab 2 report.
In total it is possible to score three points for every two weeks.


Roland: 2 points - best Lab 4 report, IDS testing solution
Maximum score is 3, nobody will be awareded more.


Hannes: 1 point - best lab 6 report
Minimum score to pass is 1.


Siim: 1 point - most popular scenario 7 solution
For those who can not score a single point, an
examination proposals will received from those
of you who scored 3 and voted.


Urmas: 2 points - lab 5 report, the set of viruses used in lab 7
== Points scored so far ==


Martti: 1 point - best lab 7 report
Margus - 3pts (lecture 1 vote, lab 2 testing method, lab 3 result)


Stefan: 1 points - best solutions developed during lab 8-1
Andres - 1pts (best lab 1 report)


Jeremy: 1 point - best solution developed during lab 8-2
Michal - 1pts (best lab 1 result)


Christopher: 1 point - best lab 8-1 report
Alexandre - 1pts (best lab 2 result)


Alexandra: 1 point - best lab 8-2 report.
Roman - 1pts (best lab 2 report)


= Scenarios =
Allan - 3pts (best lab 3 proposal with Predrag, lucky SOB scenario 4 proposal, best scenario 5 proposal)


== Scenario 1 ==
Predrag - 2pts (best lab 3 proposal with Allan, best scenario 5 proposal)
There is a Wordpress based blog.


The client is worried: How many visitors can they handle before they die.
Mauno P - 3pts (best lab 3 report, best lab 4 report, best scenario 7 proposal)


Establish a baseline: How many page displays per second AT LEAST we can handle.
Robert P - 3 pts (best scenario 4 proposal, best lab 4 result, best lab 5 result)
Disruption of the site operation is NOT acceptable.


Minimal budget!
Anis - 1 pts (best lab 4 result)


Be as detailed and technical as You possibly can.
Mart - 1 pts (best lab 5 report)


Post to the discussion group the following information:
Ronny - 1 pts (best lab 7 report)


# Detailed description of the simulation of the site. eg. if You're going to virtualize it then how? If You are running it as a process, then how You separate it from the real machine?
Mikheil - 1 pts (best lab 6 report)
# Detailed description of the attack / benchmark tools picked. Where You are going to run them? What tools? How many copies? How you are going to collect the results?!!!


=== Proposals ===
Jan - 1pts (best lab 8 result)
Benchmarking:
* AB - Discussed in lecture
* http://tsung.erlang-projects.org - Jérémy
* wget + AB - Arnis
* http://jakarta.apache.org/jmeter/ - Roland, Jérémy
* http://grinder.sourceforge.net/ - Märt
* http://httpd.apache.org/test/flood/ - Hannes
* Normal monitoring software on live server - Urmas


Simulation of the system:
Franck - 1pts (best lab 8 report)
* VirtualBox - Roland, Jérémy
* VMWare ESX - Igor
* VMWare Player - Marina
* Amazon EC2 - Märt
* KVM - Siim
* Original hardware - Hannes


=== Lab 1 ===
= Scenarios =
We will need to create two different virtual machines, one is going to be the wordpress server and the 2nd one is the client which is used to attack the server. It's probably easiest to install only one image and then clone it.


Hard disk image cloning is best done with
== Scenario 1 ==


Linux:
There is a Wordpress based blog.
VBoxManage clonehd FirstUbuntu.vdi SecondUbuntu.vdi


Windows:
The client is worried: How many visitors can they handle before they die.
"C:\Program Files\Oracle\VirtualBox\VBoxManage.exe" clonehd FirstUbuntu.vdi SecondUbuntu.vdi


After cloning the image create another virtual machine and attach the 2nd disk to that machine. As the result there will be two different virtual machines with two different virtual hard disks.
Establish a baseline: How many page displays per second AT LEAST we can handle.
Disruption of the site operation is NOT acceptable.


To be able to connect between the two virtual machines, You need to add an
Minimal budget!
additional network adapter to both hosts. It can be either "Host Only" or
"Internal" type.


To enable the new adapters in Ubuntu edit the file: /etc/network/interfaces
Assume upstream bandwidth can cope...


== Scenario 2 ==
Be as detailed and technical as You possibly can.
Design a method for evaluating any speedup solutions for the wordpress.
A repeatable process to evaluate the claims.
Whatever optimizations.
The process must be detailed enough so that somebody else can get the same results when applying that.
The "other person" is expected to have IT knowledge sufficient to install and run a Linux desktop.
The method must specify the level performance
at which the site is considered DEAD.
Budget requirements: Low - Dedicated machine + dedicated tester.


=== Lab 2 ===
=== Proposals ===
* Get Virtual machine disk image and instructions from: http://www.tud.ttu.ee/~anttix/lab2/ <br /> '''NB! DO NOT DOWNLOAD IMAGE FROM THE LINK NOTED IN INSTRUCTIONS PDF, IT WILL BE SLOW'''
* Fix bug in image (see below)
* Measure the effect of these wordpress plugins:
** [http://wordpress.org/extend/plugins/wp-super-cache/ WP Super Cache]
** [http://wordpress.org/extend/plugins/wp-cache/ WP-Cache]
* Devise Your own optimizations for the site


Bug in image: the ip address of the VM is saved in WP database as 192.168.56.3
* Install a similar system on test hardware
It is advised tu run the Target VM first and run the "sudo dhclient" command first in Target VM.
** Test hardware is smaller than
Then check active IP of target by command "ifconfig".  If it is not 192.168.56.3, then situation can be fixed by running:
* Extrapolate from current system logs
** Must have some load ...
In Target VM: "sudo ifconfig eth2 192.168.56.3"
** It may NOT be linear (it usually isn't!)
In Client VM: "sudo ifconfig eth2 192.168.56.4"
* Generate some load (without breaking it) and extrapolate from logs
* Find existing performance graphs for wordpress ... (highly unlikely)


==== A note to Windows users ====
=== Simulation environment ===
Use this command line to access VBoxManage program


"C:\Program Files\Oracle\VirtualBox\VBoxManage.exe"
* Real hardware
* Virtualization platform


==== Wordpress Admin access ====
Pick what is closer to the real system.
Username: admin
Password: QIaTemvskSeM


=== Benchmarking tools ===


==== Note to Ubuntu users ====
* httpPerf
VirtualBox with WiFi Does NOT work properly http://ubuntuforums.org/showthread.php?t=782936
* pagetop
* htop
* AB - apache bench
* curl / wget + custom scripts
* JMeter
* Grinder
* wpscan - wordpress specific


== Scenario 3 ==
=== How to separate benchmark tools from the server? ===
Your client is worried about some stuff posted on a blog. They ask You to take care of it.
They have a throwaway "script kiddie" in a third world country, who will mount the attack
so You don't need to worry about hiding the attackers identity.


Devise a way to attack a wordpress (default installation) based site to render it
* Use to VMs, Use CPU allocation in virtualization environment to allocate different CPUs for different VMs
unusable (page view times over 60 seconds).
* Put VM disk images on different disks (probably an overkill)?
* Monitor process table and make sure that test tools consume significantly less than web server processes. Lower the priority of the test tools.
* Start monitoring load and page views. Generate some load on real system. How?


Attack resources: one PC, a script kiddie, internet connection of 2Mbit/s.
How do we define the system is dead?


How are we going to evaluate the solutions?
Page view takes longer than 10sec.
* Easy to use instructions
* The most efficient use of available resources


Defacing will count, IF it works for the latest version of wordpress.
=== Determining if referenced content has an effect on results ===
* Make a list of all references the main page has
* Measure the performance of all the URLs
* Disregard the ones that load at least two magnitudes faster.


=== Lab 3 ===
=== How to measure the results? ===
Possible attacks:


* Brute force overload (AB or other benchmark), bang the slowest page possible.
* Measure response times and number of parallel requests using the benchmarking tool itself
* Connection flood (slowloris)
* Deduct the numbers from server logs
* SYN Flood (eg. with hping)
* Put the system under load, test with a normal browser if the load time is reasonable.
* Banging expensive operations (eg. adding big comments)
* etc.


== Scenario 4 ==
=== What to document? ===
Develop countermeasures for the attacks simulated in Lab 3.


Limitations:
* Exact configuration of the benchmarking tool (command line, scripts, config files etc. for EACH test performed)
* No additional hardware
* The version of the benchmarking tool used
* No reactive measures
* Virtualization platform
* Hardware specification
* URL of the page tested
* Wordpress configuration
* Wordpress content !!!
* Wordpress version
* Web server configuration
* Web server version (apache, php, sql, etc)
* The measurement results (page load times under each different test)
* Web browser used to measure response time (type, version, operating system)
* Operating system configuration (version, software installed, customizations etc).
* The AUTHOR! and the DATE!


=== Lab 4 ===
Write evertyhing down at ONCE.
if using VM-s save the image.


Evaluate the countermeasures against attacks.
=== Lab1 ===
The countermeasures must not have negative effects
on "normal usage pattern".


Normal usage pattern is defined as:
* Virtualization platform: VirtualBox
* Bursts of 10 connections per IP address in 1 second
** 512M memory, 1 CPU for each VM
* Page load times not over 20 seconds
* Wordpress 3.2.1 with default settings (no extra plugins, no performance tweaks, default skin) and default home page
* Ubuntu Server 10.04 LTS 32bit
* Web server: Apache2, SQL: MySQL (default versions bundled with OS)
* Point will go to the person that can produce the most dense graph showing the number of concurrent connections against the response time. Individual tests used to plot the data must be repeated at least 3 times.


== Scenario 5 ==
Create a solution and instructions for testing IDS
systems against the wordpress site and attacks
tried in lab 3.


A repeatable process to evaluate vendor claims.
You will need to create two different virtual machines, one is going to be the wordpress server and the 2nd one is the client which is used to attack the server.
Whatever passive IDS system delivered as a VM or a dedicated box.


Creating the IDS system itself is out of scope.
To be able to connect between the two virtual machines, You need to add an additional network adapter to both hosts. It can be either "Host Only" or "Internal" type.


The process must be detailed enough so that somebody else can get the same results when applying that.
To enable the new adapters in Ubuntu edit the file: /etc/network/interfaces
The "other person" is expected to have IT knowledge sufficient to install and run a Linux desktop.
 
Budget requirements: Modest - 2 Dedicated machines + dedicated tester + networking equipment to connect these together.
 
The process must test at least the following:
* Port scan
* SYN flood
* slowloris/pyloris
* Ab overwhelming attack
 
=== Lab 5 ===
Instructions for installing Snort IDS and its web GUI called 'acidbase': https://help.ubuntu.com/community/SnortIDS
 
== Scenario 6 ==
 
Develop/download/find/whatever a SNORT configuration (rulesets, preprocessors, whatever)
that performs better than the default configuration in lab 5 tests.
By better we mean:
* Less false positives
* Less false negatives
* The objectives are contradictory so the rule of thumb is one false negative per 10 false positives eg. solution with 10 false positives and 2 false negatives is better than the solution with 100 false positives and 1 false negatives, but the solution with 10 false positives and 1 false negative is better than the solution with 1 false positive and 2 false negatives.
* Attack is defined by a single invocation of the test script.
 
== Scenario 7 ==
 
There is a new remote vulnerability in Windows.
You became aware of a new virus that targets this vulnerability at 7 PM.
The antivirus vendor has released an updated virus database, but it's probably not installed
on all of the computers yet. Your company has 500 employees, with half of them running on laptops. You have to decide in 12 hours, if You need to collect all the employee's laptops in the morning.
 
Collecting the laptops has a high cost in lost employee productivity: There will be a queue of laptops waiting to be inspected. You can assume that all of the laptops can be inspected in half a day.
 
Limits: You can not mitigate the risk by changing the network configuration, You do not know how many laptops are still in the building. You can not take the servers offline.
 
Resources: 5 workers, some networking equipment, real servers (you can not disrupt those), "samples" of existing systems: a few spare workstations, a few spare laptops,
 
What do You know about the virus?
* It disables antivirus software
* It disables automatic software updates
* It can be cleaned up only manually in about 1 manhour
 
You need to develop an action plan:
* What You test in those scarce 12 hours?
* How?
* How do You weigh the results to make the decision?
* The plan itself must be short and easy to understand so You can write it on a whiteboard.
* Justifications for the specific actions go to a separate file/document.
 
=== Lab 7 ===
* Get viruses and XP install ISO-s from http://www.tud.ttu.ee/~anttix/lab7/
** Password for the protected zip: doomsday
* Alternatively, get Siims image
* Set up 5 VMS and connect them to a private network (or Host-Only network if You can't configure private nets):
** Plain old XP without any virus scanner (clean)
** Plain old XP with a virus scanner eg Avast (clean)
** Windows XP SP3, no virus scanner (clean)
** Plain old XP without any virus scanner (infected)
** Ubuntu Linux with tcpdump running
* If You have enogh time You can either:
** Find some more Worms and unleash them on Your virtual network (thanx to Urmas for those links)
*** http://vx.netlux.org/
*** http://www.excellentcontent.com/xbart/cbezels/
*** http://hacker-mc.blogspot.com/2008/12/pacoto-hacker.html
*** http://www.rohitab.com/discuss/oldfiles/
** Try to install Windows XP while it is connected to the infected network and see if You manage to install patches or antivirus before getting infected.
 
Measure the time it takes for the virus to penetrate the machines.
Note if virus scanner was able to catch the virus even if the underlying flaw in
windows wasn't patched.
Test if the SP3 version is immune to attacks originating from the network,
even when virus scanner is not installed.
 
== Scenario 8 ==
 
Find a solution to simulate a network of 300 PC-s in a single corporate environment, accessing a software update service at once.
 
Determine the load on primary internet connection and the router.
Assume that updates server will be able to keep up.
 
Tools to consider
* [http://www.ssfnet.org/homePage.html SSFNet]
* [http://www.omnetpp.org/ OMNeT++]
 
Additional resources
* [http://www.ssfnet.org/Exchange/gallery/index.html SSFNet module gallery]
* [http://code.google.com/p/omnet-httptools/ omnet-httptools]
 
=== Lab 8 ===
==== OMNet++ (Free for academic use only) ====
====== Install OMNeT++ ======
# Compile OMNet++ using these instructions: http://omnetpp.org/doc/omnetpp41/InstallGuide.pdf
# Run omnetpp IDE
# Choose Tutorials->Tic-toc tutorial from welcome page to open Tutorial documentation. Welcome page can be accessed from Help menu if You have accidentally closed it.
# Choose Go to workbench
# Click on tictoc in "Project Explorer" (on left hand pane)
# Choose Project->Open Project
# Choose Project->Build All
 
====== Install OMNeT++ INET framework ======
# Download https://github.com/downloads/inet-framework/inet/inet-20100723-src.tgz
# Create a directory named omnetpp-workspace and unpack inet-20100723-src.tgz in that directory.
# Switch Eclipse workspace to the newly-created directory: File -> Switch Workspace and select omnetpp-workspace.
# Follow instructions in omnetpp-workspace/inet/INSTALL file in the "If you are using the IDE" section
 
==== NS-3 (GPL Licensed) ====
Install all dependencies according to instructions [http://www.nsnam.org/wiki/index.php/Installation here] and [http://www.nsnam.org/wiki/index.php/PyViz here]
 
'''NB! Do not clone or compile anything yet'''
 
Clone NS-3 repositories
hg clone http://code.nsnam.org/ns-3-allinone
cd ns-3-allinone
./download.py -n gjc/ns-3.9-pyviz
 
Build NS-3
./build.py
 
Read the introduction slides during the build :)
* http://www.nsnam.org/tutorials/geni-tutorial-part1.pdf
* http://www.nsnam.org/tutorials.html
 
If You are unable to install the stuff, You may get an
Ubuntu VM with all of the stuff preinstalled from
 
http://www.tud.ttu.ee/~anttix/lab8/ubuntu10.04-omnetpp_ns3.vdi
 
Run Example visualizations
cd ns-3.9-pyviz
./waf --pyrun examples/flowmon/wifi-olsr-flowmon.py
./waf --run "examples/wireless/wifi-wired-bridging --viz=1"
./waf --run "examples/wireless/wifi-ap --viz=1"
 
Get itx8080 samples from:
http://www.tud.ttu.ee/~anttix/lab8/ns3-tty-v2.tar.gz
 
Run the samples
./waf --run examples/tty/lecture1
./waf --run "examples/tty/lecture2 --vis=1"
./waf --run examples/tty/lecture2
 
Look at the generated files: wantraffic*
 
Now create a new simulation that uses TCP networking instead of UDP. Refer to examples in examples/tcp.
 
== Scenario 9 ==
 
Simulate a DDoS attack against a network of web servers.
 
* 2 types of attacks
** SYN Flood
** "AB" style attack
 
Assumptions:
* 1000 "infected" hosts performing the attack
* 10 web servers, located in 5 locations
* The infected host only attacks one server
* The target to attack is selected randomly
* Assume that servers can keep up with the flood
* Data transferred during each connection is about 500kb
 
Objectives:
* Assess the bandwidth these attacks consume in each location
 
You can only implement a subset of constraints and assumptions.
Do as much as You can, maybe others have done even less :D

Viimane redaktsioon: 21. detsember 2011, kell 01:55

General Information

Schedule

Lecture TH 17:45 - 21:15 (odd weeks)

Practice TH 17:45 - 21:15 (even weeks)

Communication

Discussinons will be held at course forum in Moodle.

https://moodle.e-ope.ee/course/view.php?id=4319

All scenario proposals must be posted to their own respective forums.

Please register your account today at http://moodle.e-ope.ee

The lecturers can be reached via the course coordinator:

antti.andreimann only-people-please eesti.ee

Grading

Best of the bunch will score points.

Points will be awareded for the:

  • Best simulation proposal
  • Best result obtained in the lab environment
  • Best lab report

In total it is possible to score three points for every two weeks.

Maximum score is 3, nobody will be awareded more.

Minimum score to pass is 1.

For those who can not score a single point, an examination proposals will received from those of you who scored 3 and voted.

Points scored so far

Margus - 3pts (lecture 1 vote, lab 2 testing method, lab 3 result)

Andres - 1pts (best lab 1 report)

Michal - 1pts (best lab 1 result)

Alexandre - 1pts (best lab 2 result)

Roman - 1pts (best lab 2 report)

Allan - 3pts (best lab 3 proposal with Predrag, lucky SOB scenario 4 proposal, best scenario 5 proposal)

Predrag - 2pts (best lab 3 proposal with Allan, best scenario 5 proposal)

Mauno P - 3pts (best lab 3 report, best lab 4 report, best scenario 7 proposal)

Robert P - 3 pts (best scenario 4 proposal, best lab 4 result, best lab 5 result)

Anis - 1 pts (best lab 4 result)

Mart - 1 pts (best lab 5 report)

Ronny - 1 pts (best lab 7 report)

Mikheil - 1 pts (best lab 6 report)

Jan - 1pts (best lab 8 result)

Franck - 1pts (best lab 8 report)

Scenarios

Scenario 1

There is a Wordpress based blog.

The client is worried: How many visitors can they handle before they die.

Establish a baseline: How many page displays per second AT LEAST we can handle. Disruption of the site operation is NOT acceptable.

Minimal budget!

Assume upstream bandwidth can cope...

Be as detailed and technical as You possibly can.

Proposals

  • Install a similar system on test hardware
    • Test hardware is smaller than
  • Extrapolate from current system logs
    • Must have some load ...
    • It may NOT be linear (it usually isn't!)
  • Generate some load (without breaking it) and extrapolate from logs
  • Find existing performance graphs for wordpress ... (highly unlikely)

Simulation environment

  • Real hardware
  • Virtualization platform

Pick what is closer to the real system.

Benchmarking tools

  • httpPerf
  • pagetop
  • htop
  • AB - apache bench
  • curl / wget + custom scripts
  • JMeter
  • Grinder
  • wpscan - wordpress specific

How to separate benchmark tools from the server?

  • Use to VMs, Use CPU allocation in virtualization environment to allocate different CPUs for different VMs
  • Put VM disk images on different disks (probably an overkill)?
  • Monitor process table and make sure that test tools consume significantly less than web server processes. Lower the priority of the test tools.
  • Start monitoring load and page views. Generate some load on real system. How?

How do we define the system is dead?

Page view takes longer than 10sec.

Determining if referenced content has an effect on results

  • Make a list of all references the main page has
  • Measure the performance of all the URLs
  • Disregard the ones that load at least two magnitudes faster.

How to measure the results?

  • Measure response times and number of parallel requests using the benchmarking tool itself
  • Deduct the numbers from server logs
  • Put the system under load, test with a normal browser if the load time is reasonable.

What to document?

  • Exact configuration of the benchmarking tool (command line, scripts, config files etc. for EACH test performed)
  • The version of the benchmarking tool used
  • Virtualization platform
  • Hardware specification
  • URL of the page tested
  • Wordpress configuration
  • Wordpress content !!!
  • Wordpress version
  • Web server configuration
  • Web server version (apache, php, sql, etc)
  • The measurement results (page load times under each different test)
  • Web browser used to measure response time (type, version, operating system)
  • Operating system configuration (version, software installed, customizations etc).
  • The AUTHOR! and the DATE!

Write evertyhing down at ONCE. if using VM-s save the image.

Lab1

  • Virtualization platform: VirtualBox
    • 512M memory, 1 CPU for each VM
  • Wordpress 3.2.1 with default settings (no extra plugins, no performance tweaks, default skin) and default home page
  • Ubuntu Server 10.04 LTS 32bit
  • Web server: Apache2, SQL: MySQL (default versions bundled with OS)
  • Point will go to the person that can produce the most dense graph showing the number of concurrent connections against the response time. Individual tests used to plot the data must be repeated at least 3 times.


You will need to create two different virtual machines, one is going to be the wordpress server and the 2nd one is the client which is used to attack the server.

To be able to connect between the two virtual machines, You need to add an additional network adapter to both hosts. It can be either "Host Only" or "Internal" type.

To enable the new adapters in Ubuntu edit the file: /etc/network/interfaces