Details

Published: Friday, 15 October 2021 22:29

Hits: 732

ElectroMagnetism:

Both Electric and Magnetic fields go hand in hand. Although them seem very different from each other, one can be generated from the other. They appear as Electric or magnetic field depending on the frame of reference of the observer. We first look at Electric field and then at Magentic field.

Electric field/force:

 This force is between charged particle at rest. Coulomb's Law for Electrostatic Force is exactly similar in form to Newton's Gravitational Force. It was published in 1785 by Charles Coulomb and is given by below formula

1. Coulomb's Law: F=q1*q2/(4*Π*ε*R^2). It's similar to the formula for Newton's Gravitational law. Here instead of G, we replace it by other constant (1/4*Π*ε). q1 and q2 are 2 charges present, similar to masses m1 and m2 in Newton's Gravitational law.
2. Electric Field is defined as field due to presence of charge q and is given as E = F/q2 = q/(4*Π*ε*R^2). Electric field is measured in V/m. ε is the permittivity of the material. It measures as to how well the material itself polarizes in response to the Electric Field. This happens due to natural tendency of outside charges to rearrange themselves so that the Field is reduced. If we put the charge Q in a vacuum, then there's nothing in vacuum to neutralize the field, so vacuum will have the lowest permittivity possible. For vacuum, permittivity is ε₀ = 8.85*10^-12 F/m. Unit is F/m as ε=Q/E.1/R^2 = Cap.m/m^2=Cap/m=F/m.We take out the constant part (1/4*Π*ε) and call it the Coulomb Constant (k_e) = 1/4*Π*ε₀ = 9*10^9 N-m^2/C^2  where F =  k_e*q1*q2/R^2. Thus it can be seen that force is lot stronger than gravitational force (as k_e is very large number). 
3. Relative Permittivity: Since everything has higher permittivity than vacuum, we talk in terms of relative permittivity ε_r which is the ratio of the absolute permittivity of medium to permittivity of vacuum. i.e ε_r= ε/ε_0. A metal doesn't allow any Electric field to exist within itself, as any resultant Electric field will make electrons move (as electrons are very loosely bound), and they will keep moving until the Electric Field within the metal is no more there. So, if we put a charge surrounded by metal, metal will have no resultant Electric field, implying infinite ε. (NOTE: ε.appears in the denominator, implying higher ε.will reduce Electric field). Basically electrons will move to one end making that end -ve charged, while leaving the other end +ve charged. Thus a really big dipole is created in the metal which neutralizes the applied Electric field completely. For any material which is a perfect insulator, electrons can't freely roam whenever there's an Electric field. They are stuck in their positions bound to the nucleus. At an atomic level, the electron cloud in an atom might still get distorted. Electrons might get pushed away a little from the Electric Field creating a dipole. All such atomic dipoles will create an internal electric field in opposite direction to applied Electric field. How strong of the dipole is created is dependent on the material's ability to form such dipoles. Perfect Conductors may be thought of as an extreme case where such dipoles are created along the full length of the material, i.e electrons go on one side and nucleus goes to the opposite side. Polarizing insulators make limited length dipoles. Electrons are still held in place but just shifted a little. So, there is some net Electric field left. More the dipole electric field formed, less is the net electric field and higher is the permittivity of the material. Higher permittivity means you have to apply more charge to hold same potential difference, as some of the applied field will get neutralized. Permittivity is the capacitive ability of a material, and totally different than resistivity (see in Passive elements section). Silicon has ε_r=12, while SiO₂ has ε_r=4. This means Si will have less Electric field and hence higher ability to retain charge for same Voltage than SiO₂. Capacitors have materials known as dielectric with high ε_r so that they can hold more charge and hence more energy per unit of applied voltage.

Magnetic field/force:

This force is counterpart of Electric force. It's caused by magnetic charges instead of electric charges. The only thing unique to magnetic charges is that they always appear in pair, i.e +ve and -ve magnetic charges will appear together, and can never be separated. Usually Magnetic force or magnetic field is associated with magnets, but anything else can also behave as a magnet under right conditions. A magnet's North and South pole can be thought of as 2 magnetic charges, one at N and other at S, with opposite polarity. A moving electric charge behaves as a magnetic charge and causes magnetic field. Thus a stationary charge generates an electric field, while moving charge generates a magnetic field. However, moving is a relative term, and depends on the frame of reference of the observer. To 2 different observers in different frame of reference, the same charge can appear as moving or stationary, causing one to believe that the charge has an electric field, while to the other to believe that the charge has a magnetic field.

The equation for Magnetic field are slightly complicated, due to the fact that magnetic charges exist only in pairs. If we really had magnetic charges that could exist by themselves, then Magnetic forces would be similar to Coulomb's law. Infact, in was referred to as "Magnetic Coulombs Law".

F=qm1*qm2/(4*Π*(1/μ)*R^2). It's similar to the formula for Coulomb's law for electric charges, except that the 2 electric charges have been replaced by magnetic charges, and permittivity  ε.has been replaced by 1/μ, where μ is the permeability of the medium.

Until 1800's, this theory of magnetic charges was held. All work in early 1800's believed in the theory of magnetic dipoles (i.e magnetic charges exist in pair, creating a dipole). These dipoles gave rise to electric field. But in 1820's, it was found that magnetic field is not generated due to magnetic charges, but rather due to current, which is generated by moving electrical charges. In 1825 Andre Ampere proposed that magnetism is due to perpetually flowing loops of current instead of the dipoles of magnetic charge in Poisson's mode.

Then how do magnets generate Magnetic field, since they don't have any current thru them?

Electomagnetism:

Maxwell's Equations.

Wave Equation:

Derivation of Wave equation (Electromagetic field) from Maxwell's equations:

Very good paper here: Wave Equation derivation

Details

Published: Monday, 27 September 2021 16:30

Hits: 949

sed:

streamline editor for noninteractive purpose. This is a linux pgm that was used in older days to manipulate files. Now a days, perl and other scripts can do the job just as easily. sed allows us to do same editing as from within vi, but allows us to do it  from cmd line by providing editing cmd and name of file. It's very easy to use, and should be your first choice whenever you want to search, replace for patterns in your file. Other scripting languages require quite a bit of code to many any manipulations to a file (i.e you have to open a file, copy lines one by one to other file, after modifying lines of interest, and then close all the files. Most scripting languages don't allow editing the same file content, while sed allows you to do that). So, 10 lines of code in some other language may require just 1 line of code in sed. sed has very easy syntax (you just need to regular expression), and very few cmds to get 99% of work done.

A very good reference is here: https://www.grymoire.com/Unix/Sed.html

You can also include sed cmds in any other script, as though they were native unix cmds (as sed is installed by default on all Linux distro). So, you can think of sed as ls, cp, etc which does file editing for you.

NOTE: There are 2 variants of sed: BSD sed and GNU sed. BSD sed is the original one, but it has lot of idiosyncrasies. You should always use the gnu sed version, and that's what's installed on most Linux distro. Most of the cmds you find online are assuming GNU sed (also called as gsed). Type "sed --version" on terminal to check sed version.

Terminal prompt$ sed --version
sed (GNU sed) 4.2.2
Copyright (C) 2012 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>.

syntax:

sed <options> 'command' filename => command tells sed what to do with lines in file, change it, remove it, etc. There are many optional options that can be provided which have to be preceeded by -. The command itself may be put in single quotes, or in double quotes (see under variable substitution to see when should double quotes be used).

ex: sed -i '/_slash/d' ~/tmp.txt => Here -i is the option, '...' is the cmd, and we are using this cmd on file tmp.txt

options:

Sed cmds are provided on terminal and it prints modified o/p on screen. If we want the modified o/p to be saved in a file, we can redirect sed o/p to another file (by usng linux > operator). There are various options to change the default behaviour.

• -i => If we want to modify the original file,  we have to use -i (inplace) option. NOTE: -i is not for "case insensitive". sed doesn't have an option to be called with case insensitive option (BSD sed may have a -I option for case insensitivity applied globally, but it doesn't work on gnu sed).
• -r => sed by default uses BRE (Basic RE, see in Regular Expression section), but using -r makes it use ERE (ERE is recommended). 
• -e => To specify multiple sed cmds

command:

command follows the usual syntax: i.e 'flag1/<original_pattern>/<modified_pattern>/flag2' => This is similar to pattern replacement syntax that you see in Perl, Python, etc. "flags" in beginning or end specify what we want to do with the pattern. d=delete (used in flag2), s=substitute (used in flag1), etc. More details later.

deleting patterns: We use '/<pattern>/d' to delete matched pattern

sed -i -r '/_slash/d' ~/tmp.txt => delete _slash from tmp.txt (-i means same file tmp.txt is modified. -r means use extended regular expression (ERE)).

delete blank lines: sed '/^$/d' in.txt > out.txt => all lines in file in.txt that start and end with nothing in b/w are deleted, and o/p is passed on to new file out.txt

substituting pattern: We use 's/<orig_pattern>/<new_pattern>/' to substituting matched pattern

sed -r "s/'/ /g" in.txt > out.txt =>Here we are replacing ' with space. So, we had to use double quotes, since using single quotes gave an error => unmatched '. g is used to say do it globally.

sed -e 's/.*/PRE: & SUF \\/g' in.txt > out.txt =>Here in every line, we are replacing he start and end of line with "PRE:" and "SUF \" respectively. We have to use double backslash, since single backslash is itself an escape char. If we don't use \\, and instead use \, then / following \ is treated as literal, so we get "unterminated cmd" error.

sed -r 's/\$\{.*slash\}//g'   ~/tmp${num}.txt > out.txt => here we globally replace ${.....slash} with nothing, i.e remove that pattern. Here instead of doing inline replacement, we pass on the results to another file named out.txt

Remembering matching patterns: This is done by enclosing the pattern to be rembered using () and then recalling it back using \1, \2 etc where \1 is the 1st pattern within (), \2 is the second pattern and so on.

sed -r 's/(.*)\s+(.*)/\2 -SPACE \1/g' ~/orig.txt > ~/mod.txt => Here we are replacing all lines which have patterns of form " aa/a?aa  bbb/ccc" with "bbb/ccc -SPACE aa/a?aa". NOTE: we didn't use \(.*\) since we are using ERE (by using -r) where "(", ")" is recognized as special char. If we were using BRE, then we needed to use \(.*\)

Substituting with a variable: So far, we used single quotes in cmd section. We said that single and double quotes don't matter in the cmd section of sed. We use one over the other depending on whether the cmd itself contains single or double quotes. i.e  double quotes is used if you need to use a single quote in command itself. One other place double quotes is used is when you need to substitute the value of a var in the sed cmd. It behaves just like other scripting languages, where one kind of quote allows "var substitution" while other kind doesn't.

ex: set a ="my lord"; sed -i "s/abc.*$/ME $a/" z.log => HereAnything with "abc" until the end of line is subtituted with char "ME my lord". If we used single quotes, then "ME $a" will get substituted.

Print or delete between specific Markers: This is very used cmd when we have markers in a file and we want to cut out that protion of file, and replace it with something else. This is mostly used when some automated section of a file is updated via scripts, and we want the rest of the file untouched.

ex: sed -e '/START_MARKER/,/END_MARKER/d' File1 => d flag says to delete lines (p flag is to print lines, but doesn't work - FIXME) b/w the 2 markers: START_MARKER and END_MARKER. If END_MARKER is some pattern which is not there in the file, then everything from START_MARKER to the EOF is deleted. Use some marker as ZZZZ which is not there in file, and then contents from START_MARKER to the EOF is deleted.

Substitute text between specific Markers with contents from another file: This is not easy, as I tried several 1 liners on internet and nothing worked. The only code that works is below and it ONLY works in Bash shell. It'll error out in csh as it needs 2 lines for sed cmd to work (which apparently breaks in csh):

Link: https://superuser.com/questions/440013/how-to-replace-part-of-a-text-file-between-markers-with-another-text-file

Here, original file is edited. It has contents b/w "START of Pattern' and "END of Pattern" that are replaced by contents of replacement_file. NOTE the 2nd line on sed cmd. You have to put that code in 2nd line (i.e after a newline), else it won't work !!

lead='START of Pattern'
tail='END of Pattern'
sed -i "/$lead/,/$tail/{ /$lead/{p; r replacement_file.txt
  }; /$tail/p; d }"  original_file.txt

Details

Published: Sunday, 26 September 2021 16:10

Hits: 985

JSON:

short for JavaScript Object Notation, and is usually pronounced like the name “Jason”. We want to learn JSON syntax because it's used very frequently, takes 5 minutes to learn, and most of the websites store and send data in JSON format. XML is another text format used widely in websites for storing and sending data. JSON is better as it's faster and easier to read. A good link for JSON basics is here:

https://www.w3schools.com/Js/js_json_intro.asp

JSON defines a syntax, which can be used to store data in a text file for any purpose. JSON’s format is derived from JavaScript object syntax, but it is entirely text-based. It is a key-value data format that is typically rendered in curly braces. You can write a utility or pgm to extract key-value pair from this json file. Most of the web programming languages, as well as scripting/interpreted languages (as python, perl, etc) offer inbuilt support for parsing JSON files.

JSON is not just for Javascript, and has nothing to do with Javascript, exept that it's syntax is derived from Javascript, and so we have included it here right after the Javascript section. It may be helpful to read into material on Javascript.

ex: test.json => Here we have 3 keys and 3 values corresponding to those 3 keys. Key "first_name" has a value "Sammy", and so on. keys and values are separated by colon (:). Note there is a comma separating key/value pairs, but no comma for the last entry. 

{

  "first_name"  :  "Sammy",
  "last_name"   :  "Shark",
  "online"      :  true
}

Many webservers return data as JSON strings, which are JSON object literal or JSON array enclosed within single quotes. The 2 types of JSON strings are as follows:

1. JSON object literal: The JSON object literal is enclosed in curly braces. example as shown above. This is in key-data format.
2. JSON array: The JSON array is enclosed within square brackets. ex: ["Ford", "BMW", "Fiat"]. This is not for key-data, It's used when you have a set of data, not in key:data format.
   

NOTE: JSON strings can also be a combination of the 2 types above by having object literal nesten within array or vice versa, i.e ["Ford", {"make": "model1"}, "BMW, {"make":"model2"}]. Depending on whether that element is "object literal" or "array", we access it differently. Shown later.

To make these JSON strings, we would enclose them in single quotes as in ex here => ' { first_name": "sammy" ... } '

In JS, these json strings are parsed using JSON.parse() function which converts them into JS objects.

Even though JSON is derived from JS, there are few differences:

1. In JS or HTML files, the json object is set as a var. i.e in above ex, we can write person = { first_name: "Sammy", ... }. Then the variable person holds this js object. We access this object various ways as:
   
   • Dot notation: person.first_name or assign value as person.first_name="meme"
   • Array notation: person["first_name"] or assign value as person["first_name"] = "meme"
2. In JS, Strings always need to be in quotes( either single quotes or double quotes). But in JSON, strings always need to be in double quotes. Reason:??
3. In JS, keys can be strings, numbers or identifier names. while in JSON keys can only be strings. So, in jS keys do not need to be in quotes (except for when the keys are strings. When they are strings, we can use either single quotes or double quotes). But in JSON files, keys can only be strings and as such always need to be in double quotes.
4. In JSON, values may be string, number, another json object (can have more nested key value pair), array, boolean (true or false) or null. Values are in double quotes when they are also strings.
   1. Value as JSON object literal => { "employee":{"name":"John", "age":30, "city":"New York"} }
   2. Value as array => { "employees":["John", "Anna", "Peter"] } => This is in cases where you have more than 1 value. In this case, we don't have a way to represent these using "object literal" as they can only be in key:data format. So, we added this array format, so that all kinds of data can be represented using combo of { } and [ ].
      

Accessing JSON entries:

A very important concept in JSON is to know how to access various entries in this JSON object literal. When the JSON object has arrays and literals, with other JSON objects nested within each other. it becomes complicated. There's online JSON parsers and validators, which will show JSON files neatly formatted, so that you can access various values.

Here's one such validator: https://jsononline.net/json-validator

ex: var person =

{
  "data": {
     "current_condition": [{
        "cloudcover": "0",
         ....
         },{
        "cloudcover": "5", ...
        }]
       "past-condition": [{....}]
   }
   "info" { .... }
}

These elements can be accessed via array notation as shown:

person["data"]["current_condition"][1]["cloudover"] => This returns "5". Here 1st element "data" is "object literal" with key data format, so we access it in associative array style, i.e person["data"]. This returns the whole "data" content in "data". "data" itself is object literal in key:data format, as it's within { }. "current-condition" is the key, while data is an array, as it's within [ ]. So, person["data"]["current_condition"] points to "data" for key "current_condition". The data is an array as shown.

[{"cloudcover": "0",...},{"cloudcover": "5", ...}]

This array has 2 entries, so we refer it using array[0] and array[1]. person["data"]["current_condition"][1]points to 2nd index of array 
and is equal {"cloudcover": "5", ...}. Now this is an "object literal" with multiple entries in key:data format. We just provide the key of the data 
we are interested in. Here we do:

person["data"]["current_condition"][1]["cloudcover"] which returns data "5" for key "cloudcover"

Details

Published: Thursday, 23 September 2021 18:04

Hits: 1202

XML Extensible Markup Language

This is a specifies that says how you to arrange the data so that it's easeir for tools to read the data. The file is still a text file, but it's hareder for humans to read these files.

Here's a good link explaining the basics:

https://www.datacamp.com/community/tutorials/python-xml-elementtree

NOTE: If you run the examples in link above, do note that xml file has closing </collection> missing. If you don't add that, you will get an error when parsing the xml file.

XML:

XML looks very much like HTML files. XML creates a tree-like structure that is easy to interpret and supports a hierarchy

XML files are called XML documents