Details

Published: Sunday, 28 July 2019 20:05

Hits: 1164

wordpress:

wordpress is one of the most advanced blogging software, from which you can build almost any kind of website. Most important, it's open source, so you can see and learn how to code. It's written entirely in PHP language.

porting wordpress:

All of the wordpress files/scripts are contained within a wordpress directory, except for the database. So, when you want to migrate wordpress website from one host to another (for ex when moving from godaddy host to ipage host), it's really simple as you just need to move the whole directory and you're almost done. You still need to move the database separately, as it doesn't reside in the wordpress dir. Once you have moved the database, you will need to modify wp-config.php file in the top level wordpress dir to point to the new database. This is needed as the database location has changed. Change these values in wp-config.php file:

1. define('DB_NAME', 'new_db_name'); => If you kept new database name the same, there's nothing to modify.

2. define('DB_USER', 'new_db_user'); => If you kept new database user name the same, there's nothing to modify.

3. define('DB_PASSWORD', 'new_password'); => enter your new password here.

4. define('DB_HOST', 'name.dbxx.x.abc.com'); => Critical that you modify the sql hostname to match the new host name. You can get the new database hostname from the web hosting provider. Another way to find this

At this point, we are done, migrating wordpress. A very good tutorial on this is HERE .

wordpress structure:

In order to understand basic structure of wordpress, I've listed main dir and files:

1. WEB-INF dir:

2. wp-admin dir:

3. wp-content dir:

4. wp-includes dir:

5. Numerous php files:

index.php is the first file that is accessed in the wordpress dir whenever a visitor visits the website. If you recall, index.php or index.html are the default files that the browser looks for when the url is typed (www.abc.com), with no files specified in the url (www.abc.com is equivalent to www.abc.com/index.html or www.abc.com/index.php).

Theme:

I have used default "Twenty Ten" theme. It's named Twenty Ten, since it was released in year 2010. Every year a new theme is released by Wordpress team named after that year. Currently for year 2020, we have "Twenty Twenty" as the theme.

The very first thing you might want to do is to customize the appearance of this theme. This can be done by going to administrator panel, and choosing Appearance tab on left side. Then click on Themes.

Your current active theme shows up. Click "Customize" on it. A new screen shows up. Click on "Header Image", then "Add new image" and upload your new image. Don't worry about the dimensions of the image, as the menu will allow you to crop the size of this new image to fit on your website. Once Published, the new image will show up on the top bar of your website.

You can further customize what gets placed on what side of page via "widgets". Instead of clicking on "Themes", click on "Widgets" on Appearance tab. You will see lot of widgets for images, archives, calender, etc. You can choose your desired widegts and drop them on right side to wherever you want them to be placed. I placed my pictures and search box, this way, on right side of page.

One good way to learn about themes is to create one yourself from scratch:
good tutorial here: http://webdesignerwall.com/tutorials/building-custom-wordpress-theme

Details

Published: Thursday, 25 July 2019 18:25

Hits: 1509

Emacs:

emacs is open source editor to edit or create any text file. It's licensed under GPL. It's written in Lisp, and it's functionality can be extended by anyone, by writing macros in Lisp.

emacs can bw used to write programs in any language. Beauty is that it allows automatic indenting, braces, etc needed for that particular language, so that you can identify your mistakes even while typing.

Xemacs is a branched version of emacs, derived from emacs 19. Their features are roughly comparable, but they differ significantly at Lisp programming level. Xemacs is not currently being actively developed, so it's behind emacs in terms of features. So, stick with emacs, as there is nothing you can do on Xemacs and not on emacs. Plus emacs is the real gnu project.

A lot of good doc for emacs is on gnu website: https://www.gnu.org/software/emacs/manual/

especially the emacs pdf manual: https://www.gnu.org/software/emacs/manual/pdf/emacs.pdf

Remember, emacs is huge, so don't spend rest of your life reading all the manuals. Instead, just 10-20 cmds will suffice to get most work done. Then, you can search for additional cmds, as you need them.

Install:

You should always install GNU emacs GTK+ version, as other version of GNU emacs don't have graphical buttons for browsing thru dir, etc. This is usually referred to as "Emacs metapackage", and it's guaranteed to to be the latest recommended Emacs variant.

linux mint: sudo apt install emacs => installs latest emacs, as of 08/2019, it's emacs-25.

Ubuntu: Under Synaptics package manager (on Ubuntu Cinnamon), look for emacs (In description, it should say metapackage). Install it. Or run "apt search emacs metapackage" on cmdline in Terminal, and then install the right one. On my system, emacs got installed at /snap/bin/emacs, instead of at /usr/bin/emacs.

Settings:

When you type "emacs" or click on "emacs" shortcut, it brings up emacs gui based on default settings. You can customize the looks and feel of emacs based on your preference. One of the files that is used for this is ".emacs" in your home dir in linux. Anyfile with a "." in the front is a hidden file, so it doesn't show up via ls, but shows up as long listing, i.e cmd "ll".

If you don't already have a .emacs file, create one.

Syntax:

• comments are written using ";;" at beginning of line, or via ";" at end of line.
• All custom settings for variables are put in (custom-set-variables '(<SETTING1> <VALUE1>) '(<SETTING2> <VALUE2>) '(...)). We can use separate stmt for eah setting, but the syntax has to be followed.
• All custom settings for faces are put in (custom-set-faces '(<SETTING1> <VALUE1>) '(<SETTING2> <VALUE2>) '(...)). 

1. show name of buffer: One of the important settings is where you want the name of the file to show up on the title bar. By default, emacs shows user@hostname on title bar. This becomes an issue when you have too many windows of emacs open, as you can't see at a glance which file is open on which window. If you usually keep more than 1 emacs open, it's best to add the below setting so that it shows the name of current file open in title bar.

;; add this so that titlebar shows name of buffer
(setq frame-title-format "%b")

2. inhibit annoying startup screen: You can use below settings for that:

(custom-set-variables
 '(inhibit-startup-screen t))

3. Display line numbers permanently: You can set this via below setting:

(custom-set-variables

'(global-display-line-numbers-mode t))

4. init file: prevent emacs from loading init file from xeamcs

(custom-set-variables

'(setq load-home-init-file t)) ; don't load init file from ~/.xemacs/init.el

5. verilog setting => not sure for what ? FIXME ?

(custom-set-variables
 '(verilog-typedef-regexp "_t$"))

Save settings automatically:

Instead of typing these settings in .emacs file, we can automatically save these settings by going to options->save_options in top menu of emacs. This will add pertinent lines to .emacs file automatically based on what you have currently.

Ex: For displaying lines, we goto: Options->ShowHide->Line_Numebers_for_all_lines->Global_Line_Numbers_Mode. This starts displaying line numbers. Now if we goto Options->Save_options then this option " '(global-display-line-numbers-mode t) " will be added to .emacs file. This way we don't have to remember all settings manually.

run emacs:

emacs /home/file.txt => opens file using emacs gui

commands in emacs:

emacs has a gui with buttons for file, edit, etc. It's very easy to use. Each button also has a shortcut. Any cmd that can be executed using buttons can be executed using short cut too. Common shortcuts here: https://shortcutworld.com/Emacs/linux/Emacs_23.2.1_Shortcuts

In shortcuts in above link, Meta key (M) refers to "Meta" key that was present on MIT keyboards where emacs was developed. Out side of MIT, Meta key didn't exist on Keyboards. So, 2 alternatives key were chosen => Esc and Alt key. Meta was mapped to both keys. Usually it's the left hand Alt key that it was mapped to. Since Alt key may not be present on all keyboards, emacs allowed Esc key also to be used as Meta key. On Mac notebooks, M key refers to "command" key (as command key is a substitute for Alt key). However, the way we use Esc key and Alt key for Meta key is different. With Alt key, we keep it pressed, while pressing other keys. With Esc key, we have to press Esc key, then let it go, before we press other keys. Generally, it's safe to stick with Alt as Meta key, since that always works.

Similarly Ctrl key (C) refers to "control" key on bottom left of keyboard.

When it says "ctrl + s" key, it means press 1st key (here it's "ctrl key") and then while it is still pressed, press the other key (here it's "s key").  When it says 3 keys as "Meta + Ctrl +s", it means first press Esc key, then let it go. now press "ctrl+s" in regular fashion.  When shortcut keys show comma in b/w, as in "ctrl + x, 1", it means press "ctrl+x" in regular fashion, then let your hands go off those keys, now press the 3rd key after comma ("1 key").

Built in functions: All these shortcuts are actually executing built in functions within emacs. We can run those function directly too. The way to execute function is "M + x" (Meta  and  "x" key where "x" is fr execute) followed by typing the function name.  Ex: To run function "forward-sexp" you need to hit Meta and x, keys and then type "forward-sexp" on the cmd line.

1. search:

emacs has a button for "search" under "edit" menu. To do simple string search, we do "string forward" or "string backward" (can also be done by using shortcut "ctrl + s" keys, i.e "ctrl" key followed by "s" key for forward string search, or "ctrl + r" for backward string search). Now, if we keep on pressing "ctrl+s", we will go to next searched item for that string, and continue going, until we are at last searched item.

emacs supports RE (not sure if ERE supported). More details in emacs section. Link: https://www.emacswiki.org/emacs/RegularExpression

So to do regular expression search, which is much more powerful, we need to do "regexp forward" or "regexp backward". This has a 3 key shortcut = "Meta + Ctrl +s". Pressing meta key changes meaning from string search to regexp search. Now we can press "ctrl+s" key, and it will start doing regexp search, with whatever regex pattern we provide. Now, if we keep on pressing "ctrl+s", we will go to next searched item for that regex, and not to the string. However, if somehow we lose the Esc key (because we pressed any other key or pressed on text), then pressing "ctrl+s", will bring us back to regular string search. We have to press "esc" again to get into meta mode. Then pressing "ctrl+s" will get us to regex search, with last searched regex pattern.

ex: press Meta+ctrl+s, type .*(me) => This searches for pattern "(me)" in all the lines. Since it's BRE search, we don't need to escape ( ) as by default parenthesis ( ) are treated as literals in BRE.

2. search and replace: This works the same way as search, except that it searches and replaces the matching pattern with your new replacement pattern. emacs has a button "replace" under edit menu. This is interactive search and replace, which replaces patterns one by one waiting for your yes/no (y or n) answer before each replacement. Instead of typing y or n, we can type "!" mark (exclamation mark without quotes), and it will search and replace globally

Link: https://www.oreilly.com/library/view/learning-gnu-emacs/1565921526/ch04s02.html

Just like in "search" option above, we have 2 flavors here => one is regular string "search and replace", while other is "regex" search and replace.

ex: add a / at the end of each line. For this we need to use "regex search and replace". To do this, first choose from top Edit->replace->replace regexp. Then type "\(.*\)$" (without the quotes), hit enter, and then type the replacement pattern, which is "\1/" (without the quotes). Since emacs is BRE, ( is escaped with backquotes to treat it as special character. We store everything before the end of line in (.*), which is later accessed using \1, then we put / at end. This pattern is useful as many times we want to add a comment or special char at end of each line.

ex: replace [text] with \[text\]: Again use regex for this. search = \[\(.*\)\] replace = \\[\1\\] . [] is BRE metacharacter, so escape it using \. () is treated as literal in BRE, so we escape it to be treated as metachar. So, this finds everything within [] and stores it in \1 as we used () which stores whatever matched inside it. Now we replace [ with \[. But \ by itself is escape char, so use \\ to treat it as literal. [ doesn't need to be escaped as it's understood to be a literal. []  is not treated as metachar as [] within a replacement string doesn't make sense as a metachar (i.e [abc] in search means match any of "abc" but in replace it would have no meaning). We can test this by trying to put \ infront of [ (i.e using \\\[\1\\\] would error out with msg "invalid use of \"). So, what we have works w/o issues. Finally try some text. As an ex: [5ty] will become \[5ty\]

3. select, cut/copy/paste:

To select a large region of text, it's inefficient to select start of text and keep rolling the cursor. A better way is to press CTRL + SPC (i.e "ctrl" + "space" key) with the cursor at the start of text that we want selected. This sets the mark at one end of text. Then we goto end of text,  and everything until that point is selected. Now we can use cut,/copy buttons to cut or copy the text. Then we can paste this text anywhere using paste button.

4. comment out regions of code:

Sometimes, you write a pgm using emacs, and want to comment out a large section of code. To do this, you first select the region to comment out, then press Meta+x keys (known as M-x), and then type comment-region in that bottom section where M-x appeared. Instead of writing long cmd "comment-region", we can also write shot form "comm-r". Then on pressing enter, all of the text within hselected region will be preceeded by a comment character. However, emacs may not know what is valid comment character for this particular file, so it will ask you to provide a comment character (i.e for python and many scripting languages, it's #). Once a comment character is provided, from next time, it won't ask you for a comment character.

Another way to comment or uncomment a selected regios is to press shortcut keys "Meta" key + ";" key (i.e press Meta and then ; while keeping Meta key pressed). This will comment out the whole selected region. If the region was already commented, then it will uncomment it. Other shortcut is press "ctrl + x" and then "ctrl + ;". This will comment or uncomment. However, uncomment sometimes didn't seem to work with these set of keys, and it ended up double commenting. Not sure what causes this indeterministic behaviour.

5. Finding/Matching balanced expressions:

Sometimes, you want help finding where the piece of code starts and where it ends. These are usually put in brackets, parenthesis, braces (called parenthetical group or PG), or may be words, numbers, etc. We can navigate thru these by using key combo:

1. PG: we can use keys "M + C + n" (which is ctrl+alt+n key) to move to next one to closing PG, or " M +C + p" to move to previous one or opening PG.
2. Balanced Expressions: Instead of using n or p keys, we use "f" and "b" to move forward or backward.
3. Built in function: We can directly execute function "show-paren-mode" to highlight parenthesis. Hit "Meta" and "x" keys followed by typing function name "show-paren-mode"

variables:

Emacs uses variables internally to determine various settings. You can change those globally, or for just 1 buffer. A variable is a Lisp symbol which has a value. The symbol's name is also called the variable name. A variable name can contain any characters that can appear in a file, but most variable names consist of ordinary words separated by hyphens. There are various ways to read/change these variable values, or define new variables using gui. However, we can specify these variables within a file also, so that when we open the file using emacs, these var will get applied, and that file will open consistently with same looks and feel for anyone opening that file. However, there is a problem with this approach. These extra lines specifying emacs var values are not part of that file's syntax and will probably cause a syntax violation for that language, when run with a simulator. So, we put these var within comments for that kind of file (i.e if it's C pgm, we put it within /* */), but emacs is able to interpret that these var within comments are meant for it.

There are 2 ways to specify such var:

1. in first line of file, using "-*-" line: If first file specifies an interpretor(i.e #!/usr/bin/perl), then put this line as 2nd line of file. You can specify any number of variable/value pairs in b/w -*-, each pair with a colon and semicolon (i.e -*- var_name: value; ... -*-). NOTE: such lines are preceeded by comment syntax for that kind of file.

ex: In perl file, test.pl, we add below line as 1st or 2nd line. keyword "perl" in b/w -*- specifies to emacs that this file is perl file. So emacs sets it's display based on perl. NOTE: this line is put within perl comment "#", so that perl interpreter ignores it. Here, we do not have "var_name: value" format, not sure why? FIXME ?

#-*-perl-*-

ex: In tcl file, test.tcl, we add this line. This specifies various var_name and their values. special var name "mode" specifies major mode, here it's "tcl". That implies to emacs that it's a tcl file.

# -*- mode: tcl; basic-offset: 4; -*-

2. specify local var list at end of file: Emacs identifies anything within string "Local variables:" and "End:" as emacs local variables. In between these strings, come the variable names and values, one set per line. The set are specified as "variable_name:value", so that var name is assigned that value.

ex: In verilog file, test.v, we can add following lines at end of file. So, verilog var are assigned values as indicated. NOTE: all these lines are put within comments (i.e // or /* */)

// Local variables:

// verilog-library-flags:("-f vexpand.dirs")

// verilog-auto-inst-param-value:t

// End:

The above 2 methods are for specifying per file var, but we can also specify these var on per dir level, so that it applies these var to all files, within a particular dir. We can also do this on per buffer level. This is all explained here: https://www.gnu.org/software/emacs/manual/html_node/emacs/Variables.html

AutoSaved Files during Crash of emacs: whenever emacs crashes or your computer shuts down, emacs saves a file with same name but with #. i.e myfile is automatically saved as #myfile#. Since it saves this file periodically, you can get most of your edits. You have to open your file "myfile", and then type M-x (Esc key followed by letter x), and then type recover-file. emacs will guide you to recover the file. You have to enter the file name you want to recover (i.e myfile), and then it will auto prompt with that it will use to replace this file (i.e #myfile#). If this looks complicated, you can just copy #myfile# as "myfile" and all your last changes will get saved in original file.

Various modes for emacs:

emacs can have various modes to support writing your program in different languages. This support makes it easy to see visually different sections of code, identify syntax errors, missing parenthesis, etc. See  Verilog section for verilog mode of emacs. Other modes exist for languages as Python, C, tcl, bash, csh, etc. Now-a-days, IDE (Integrated Development Editor) do these things where they show the code written in a particular language very nicely formatted, colored and visually appealing.

Details

Published: Monday, 08 July 2019 19:28

Hits: 1183

When building any website, we need to know about 2 things:

1. language to code your webpage in = usually php

2. language to build a database of objects = usually mysql (mysql is really a client/server software, which uses SQL language to allow client to talk to server)

Think of database as a big file that stores all your data. Usually this data gets very big (i.e a website that has a list of all items, their price, description, etc). If we store all this data in 1 big file, then just reading or writing to this massive file every second will be very slow. This is because appr location in file needs to be searched for and then data written over there.  Also not feasible, since updates will be happening to this file like every micro second, whenever a customer places an order, or prices change on items, etc. This will make operations to file very slow. So, a special software comes to our rescue called database management software (DBMS). DBMS still store data as one big file (we have options to break it in smaller files), but their real value comes from their ability to be able to do quick searches, maintain transaction atomicity, prevent datafile from getting corrupted, etc.

There are many varieties of DBMS. One of them particularly popular is RDBMS (Relational DBMS). This is called relational database because all the data is stored into different tables and relations are established using primary keys or other keys known as Foreign Keys.

In DBMS terminology, database refers to a collection of tables with relation b/w them. Tables are just a simple data matrix (like excel sheet) with rows and columns.

MySQL: MySQL is a fast, easy-to-use RDBMS. It's open source, and can support large databases (upto millions of Terabytes).. SQL is a data language, and MySQL uses that language to access the database it stores. MySQL cmds can be accessed via functions in PhP, Perl, Python, etc, so it's really easy to work with MySQL, w/o even explicitly learning SQL cmds. MySQL is wriiten as capital M, S, Q, L, and small y, but below we'll just write it as mysql frequently to promote my laziness.

MySQL pgm has client server architecture. This is very common for pgms, which don't always run on local m/c, and can be accessed from anywhere. MySQL pgm has 2 main pgms => server pgm and client pgm. client pgm runs on local m/c when you type "mysql" on unix promot. Server pgm (called mysqld) runs automatically whenever the m/c is started (provided mysql is installed). As you can see server and client pgm can run on any m/c, and don't have to be the same m/c.

MySQL: good doc here: https://www.tutorialspoint.com/mysql/mysql-quick-guide.htm

One other more comprehensive place is: http://www.mysqltutorial.org/

Offical mysql website also has decent tutorials: https://dev.mysql.com/

Install MySQL: The below link on mysql offical website shows steps in detail on how to install it for apt, yum, etc.

https://dev.mysql.com/downloads/

These are the cmds for installation on linux mint (or any linux derivative using apt):

1. sudo apt-get update

2. sudo apt-getinstall mysql-server => This installs mysql server, mysql client and many other files. Here it installs distribution=5.7, which may not be the latest (there are additional cmds listed which will get you the latest). We do not need to install both client and server, as many times, we don't even need client/server on same m/c. So, we can choose and install only the part that we need. For our purpose, we install both client and server on our local m/c.

After installation, mysql server automatically starts. These are the 2 main binaries that get installed:

A. mysqld => server pgm that contains database. It manages access to the actual databases on disk or in the memory. Binary is /usr/sbin/mysqld. When MySQL server starts, it listens for network connections from client programs and manages access to databases on behalf of those clients. If we see list of processes running (using ps -ef), we will see mysqld running

ex:  /usr/sbin/mysqld --daemonize --pid-file=/run/mysqld/mysqld.pid => "ps -ef | grep mysqld" shows mysqld daemon running with shown args. mysqld has many options that can be specified at startup.

B. mysql   => client pgm (cmd line) that connects to mysql server to manipulate the information in the databases that the server manages. Binary is /usr/bin/mysql. This doesn't start automatically. This is the pgm that we invoke all the time (we won't talk about mysql server pgm anymore, as it's not relevant to most users). Whenever we say "mysql", we almost always means mysql client pgm.

After installation, verify that mysql is installed:

mysql --version => shows mysql client version = mysql  Ver 14.14 Distrib 5.7.27, for Linux (x86_64)

mysqld --version => shows mysqld client version = mysqld  Ver 5.7.27-0ubuntu0.18.04.1 for Linux on x86_64 ((Ubuntu))

mysqladmin --version => shows "mysql admin" script version = mysqladmin  Ver 8.42 Distrib 5.7.27, for Linux on x86_64. mysqladmin is additional pgm included with mysql package to simplify admin tasks.

mysql --help => shows all options for mysql client pgm

MySQL server allows only certain users on certain machines to connect to it for security purposes (just like ssh, etc). It has a list of usernames and passwords stored in one of it's table, along with the name of host machines from where they can connect, and only those users are allowed to connect. Else we get "Error: 28000: access denied" from the server. During mysql installation, a "root" user account with no password on host "localhost" is defined and added to this table.This means a user  "root" on local machine (or on ip addr: 127.0.0.1) can connect to mysql server on that machine using mysql client. Any other user from any other m/c is not allowed to connect to mysql server. So, if we want to run "mysql" client pgm on the same m/c to connect to mysql server, we need to change user to "root" on our machine, meaning we need to use a "sudo" cmd preceeding mysql cmds (or run "sudo -i" to change to root, and then use cmds w/o sudo. su wouldn't work as root a/c is disabled by default in many linux distro for security reasons),

So now we know, that anyone logged in the m/c as root, or using cmds with sudo in front (which gives them same rights as root) can connect to mysql server. mysql server by defaut doesn't require a separate password for root user  to connect to it. This is risky. To fix this, we'll set  password for root (for mysql server user "root").

If we look in /etc/passwd file, we will see list of all users on that machine. We'll see user "root" and user "LOGIN_USER", along with other users. "LOGIN_USER" here is the name of username with which we log into our linux machine (for ex: for username "Rakesh", LOGIN_USER is Rakesh, so we'll see username "root" and "Rakesh" in passwd file). After installing mysql, we also see a new user "mysql" added to this file, as mysql pre-install script adds a user "mysql" using "adduser" cmd.

The mysql.user grant table defines the initial MySQL user account and its access privileges. Installation of MySQL creates only a 'root'@'localhost' superuser account that has all privileges and can do anything. If the root account has an empty password (which is the default), your MySQL installation is unprotected: Anyone can connect to the MySQL server as root without a password and be granted all privileges (although only on local m/c). So, we assign a password to "root" user using any of 2 scripts below, before invoking mysql client. NOTE: we use "sudo" before each cmd, since only "root" user is allowed connecting to mysql server.

1. sudo mysql_secure_installation => this brings up the script to tighten up security for mysql. Most important thing to enter here is your mysql client "root" account's new password.  More details here: https://www.computerbeginnersguides.com/blog/2017/08/02/install-and-configure-mysql-server-5-7-on-linux-mint-18-2/

2. sudo mysqladmin -u root password "new_password"; => this gets new passowrd assigned to "root" a/c on mysql server

cmds to start/stop mysql server: (we use service cmds here, check in linux server notes). All cmds preceeded by sudo.

1. sudo service mysql status => shows mysql is active (running). As mysql automatically starts running on installation

2. sudo service mysql stop => stops mysql server

3. sudo service mysql start => starts mysql server. For restart, we cn also use "mysql restart"

starting MySQL client:

You can connect to your MySQL server through the MySQL client by using the mysql command on cmd line. We have to specify the sql server m/c ip addr (or name), user name who is connecting to it and password for that user. If you haven't assigned any password to "root" account on mysql server, then you can connect to mysql server w/o giving any password as by default it will be set as blank. Also, we'll need to connect as user "root" since no other user is allowed to connect.

> sudo mysql => typing this makes mysql monitor for client appear, and it connects to mysql server locally (Since default host name is localhost). The default user name is unix login user, but by typing "sudo", we become "root" for purpose of executing that cmd. This logins as user "root@localhost", as sudo changes user to "root". IF we don't use "sudo" or are not "root" user, well get access denied error, as user will be unix login user, who doesn't have access rights for mysql server.

cmd line options for mysql client:  Most of the cmd line options can be given in short form as "-o <NAME>" or in long form as "--option=<NAME>". We can also specify all these options in an option file instead of typing them on cmd line.

-h <hostname> => specifies host name, default is localhost. Can also be written as --host=<hostname>

-P <port_number> => specifies which port on the host to connect to. NOTE: capital "P" Mysql server pgm is bound to TCP port number 3306 by default. Since MySQL server can connect to any remote client, it has to be assigned a port number from which it grabs all it's input. This is true for TCP/IP protocol which works that way. It can be any one of 65K ports assigned for connections using TCP/IP. This port is just a chunk of memory (buffer) where incoming data from internet connection is dumped so that specific pgms continuously monitoring this port can start consuming data in this buffer. This port buffer will have additional info on which ip addr the data is coming from, user name, permissions, etc for mysql server to use. That is why when we access mysql server locally, we would expect it to be coming from an ip addr 127.0.0.1(i.e  "localhost"), from port 3306. However in reality, localhost doesn't use TCP/IP protocol, and instead uses unix socket file to transfer data. Socket file are special files used for Inter process communication by the OS. API for unix socket is similar to that of Internet socket, but all communication occurs entirely within kernel, and filenames are used as addr instead of IP address (since communication is all local). So, -P has no meaning for localhost and is ignored.

-u <username> => specifies username, default is unix login user

-p<passwd> => specifies password for that user, NOTE: there is no space b/w -p and password. Also, if we use just -p with no password, then mysql prompts for password on next line, which is safer, since password is not visible in plain text. Also, all processes along with their options can be seen by typing "ps -ef" by any user on that m/c, so anyone can see user's password, if password was entered on cmd line. So, always use "-p" with no password. 

<db_name> => we can optionally specify name of database to open (with no - option) that we want to open. If we don't specify database name, we connect to sql server but can't access any database, until we select a database using "USE" SQL cmd on sql client prompt.

ex:

> mysql -h example.com -u rajat -p => here we connect as user "rajat" on host "example.com on default port 3306. If rajat is not an autorized user in mysql server table on that host, then he can't connect. Since no password provided after -p, then password is prompted on net line.

Enter password:*******

mysql client prompt appears at this time, and we can start typing mysql cmds.

 mysql dir structure: After installation, we'll have many dir with mysql files in them. Details of source code and dir structure is here (very good descr, should read this): https://dev.mysql.com/doc/internals/en/

Here is how the flow works:

1. User invokes mysql client by typing "mysql" with appr options. client connects to the server, and does initial handshake. Encyption and authentication methods are established (SSL can be used for both encryption and authentication).
   • In a classic client/server, the server has a main thread which is always listening for incoming requests from new clients. Once it receives such a request, it assigns resources which will be exclusive to that client. In particular, the main thread will spawn a new thread just to handle the connection. Then the main server will loop and listen for new connections.
   • MySQL has a great variety of mutexes (mutual exclusion object) that it uses to keep actions of all the threads from conflicting with each other.
   • For each thread, there is an infinite loop. The loop repeatedly gets and does commands received from sql client. When it ends, the connection closes. At that point, the thread will end and the resources for it will be deallocated.  
   • All communication b/w client/server is done via packets. Packets are formatted messages that client/server send over tcp/ip lines. Packets will have a header, an identifier, and a length, followed by the message contents.  
2. User types any SQL cmd on mysql cmd prompt. client performs initial checks. Source code for this sits in client dir of mysql s/w. As an ex, we enter SQL cmd INSERT =>  mysql> INSERT ,,,
3. The mysql client then passes this edited/checked cmd over TCP/IP to sql server @ip_address and to default port for sql server. If server is local, then ip addr is 127.0.0.1, and file socket used. These cmds are passed in form of  packets.
4. Then via the vio routines (in C), the server grabs this cmd, Low level mysql routines are called to parse sql (source code in sql dir), and doing what's necessary (Source code in mysys dir)
5. Finally, one of the ha (handler) programs in the sql directory will dispatch to an appropriate handler for storage. Depending on the storage engine, handler may be in myisam or Innodb dir.

Few of the important dir are:

1. /usr/bin, /usr/sbin => These directories contain mysql/mysqld binaries

1. /etc/mysql/ => This has many configuration files. Most important is database server configuration file: /etc/mysql/mysql.conf.d/mysqld.cnf. This file has settings for where all log files, data files etc are stored. Data file for our database is stored as 1 big file here:  /var/lib/mysql/ibdata1

2. /var/lib/mysql => this dir has all logs, data, etc. We see ibdata1, which is the some data file that stores databases. More data files would be named ibdata2, ibdata3, etc. Note sure what they store ?? FIXME??. Inside this dir, we see subdir for all databases (as mysql, sys, etc). If we create a new database, a new dir with that name gets created with file db.opt in it. It is a 2 line file, with charset defined for that db. When we create new tables using "create table customer" then 2 new files for that database get created - customer.frm and customer.ibd. These are both binary files.Each table has it's own .frm and .ibd files, along with lot of other files. In other database dir as "mysql", we see lot of other files.

.frm => every MySQL table you create is represented on disk by a .frm file that describes the table's format (that is, the table definition). We can do hexdump to see contents of this file. It shows header which stores various info about the table.

hexdump -v -C customer.frm

.ibd => These store actual data, i.e table values inserted. Data is stored in compact byte form, Each data assigned storage space in file based on it's data type. This is much larger file, as it stores data.

SQL cmds:

We can type SQL cmds directly on mySQL client prompt, or put all the cmds in a file, and then run that file on mySQL client.

• Each SQL cmd needs to be terminated by ";".
• SQL is not case sensitive, but convention is to hav capital letters for SQL cmd and small letter for arguments.
• 3 style of comments in SQL. # and -- treats anything from # or -- to end of line as a comment. -- needs to be followed by atleast one whitespace for it to be recognized as a valid comment. C style /* ... */ is used for in line as well as multiline comment.

1. Databases: A mysql server can keep several databases for different websites or projects, all in same server. This is accomplished by giving separate names to databases. Each of these databases can have multiple tables to store the actual data. The complex security mechanism ensures that individual databases and tables can be accessed only by authorized users. That is why we have to provide a username when starting mysql client, so that mysql client can check which databases and tables are allowed to be accessed by that user. Database related cmds:

- SHOW DATABASES; => shows all databases.By default, mutiple databases are already there. "mysql" is one of the databases that is automatically built on installation that stores user info.

- USE mysql; => this selects a database to work with. This cmd is essential to start working with database, else no database is selected, and so no database related SQL cmds will work. We can also select a database when connecting to mysql client via -D option, i.e mysql -u robert -D mydb -p => selects mydb database
 
- SELECT DATABASE(); => This shows all selected databases.

- CREATE DATABASE test_db; => creates a new database named test_db, This creates a dir named test_db in /var/lib/mysql. Inside test_db will be a *.opt file which stores various configuration for this db. We can only select 1 database at a time, so "USE test_db;" will select this database instead of any previously selected one. In mysql, SCHEMA is the synonym for DATABASE, so they can be used interchangeably.

- DROP DATABASE test_db; This deletes the said db.

 

2.Tables: Tables are most important part of a database. Here it is where all all the data in a database is finally stored. There are various table types also called storage engines that you can choose from. These storage engines are internal way used by a database server to store data. Uuually we should not care about storage engine, as we do not need to change it. The default storage engine from version version 5.5 an above is InnoDB, while before that it was MyISAM. Besides these, there are many other storage engines as MERGE, ARCHIVE, CSV, etc.

A. InnoDB: The InnoDB tables fully support ACID-compliant and transactions. They are also optimal for performance. InnoDB table supports foreign keys, commit, rollback, roll-forward operations. The size of an InnoDB table can be up to 64TB.

B. MyISAM: The MyISAM tables are optimized for compression and speed. However, they are not transaction safe.

Tables store data as a table with rows and columns. We specify how many columns each row has, what is the data type that each entry in a particular column, and that's it. The data types are of 3 categories:

I. Numeric: This includes INT (signed or unsigned upto 11 digits), BIGINT (upto 20 digits), FLOAT, SMALLINT, etc.

II. String: This includes BLOB or TEXT (max of 65K char), VARCHAR (from 1 to 256 char, num of char needs to be specified in brackets as VARCHAR(5) means 5 char), CHAR (CHAR is a string with default char length as 1, but can be defined as CHAR(5) indicating it can have up to 5 char), LONGBLOB or LONGTEXT, TINYBLOB or TINYTEXT, ENUM, etc. The difference b/w BLOB and TEXT is that sort and comparison on BLOB data is case sensitive, but not so for TEXT. BLOB is Binary large object and is used to store large amount of binary data too. ENUM is enumeration or a fancy term for list. Just as in other languages, it allows only those values put in the list to be allowed as valid. ex: ENUM( 'ABC', '12') will allow only ABC, 123 and NULL as valid values for that field.

III. DATE and TIME: This includes DATE (stored in YYYY-MM-DD format), TIME (stored in HH:MM:SS format), DATETIME (stroed in YYYY-MM-DD HH:MM:SS format,)

NOTE: NULL is a special data type which represents "nothing", and it can be applied to any of 3 data types above, whenever we do not want to store a value. Any of above 3 data types above can have NULL value.

These are few table related cmds.

1.CREATE TABLE table1( ... ); => This creates tables with specified columns. Each column has data name, data type and optional data attributes. ex:

CREATE TABLE customer (

Idx INT AUTO_INCREMENT, #This is 1st col. It has field called "Idx" which is of type Integer, and it's value is auto incremented, whenever a new row is created. AUTO_INCREMENT is an optional attribute. Field Attribute AUTO_INCREMENT tells MySQL to go ahead and add the next available number to the id field.

name VARCHAR(236) NOT NULL, #VARCHAR(236) says it can have upto 236 char. "NOT NULL" attr specifically states that "name" is not allowed to have "NULL" value, so if we try to create name with "NULL" value, mysql will raise error.

start_date DATE,

description TEXT,

PRIMARY KEY (Idx) #NOTE: no comma for last field. Keyword PRIMARY KEY is used to define a column as a primary key. You can use multiple columns separated by a comma to define a primary key.
)  ENGINE=INNODB; #Engine name is optional. NOTE: semicolon at end of this line to indicate end of SQL stmt. So, above cmd is 1 long cmd spread on multiple lines for clarity.

2. INSERT INTO table1 (field1, filed2, ...) VALUES (value1, value2, ...); #after creating tables, we use this cmd to insert values into fields of table. We use this cmd repeatedly to insert as many rows as we want to the table. ex:

INSERT INTO customer (description, name, start_date) #Idx is not inserted into table, as Idx has AUTO_INCREMENT attr, which means it's incremented by 1 and assigned to "Idx" for this row, whenever we insert a new row.

VALUES ("Learn PHP", "John Poul", NOW()); #Here NOW() is a function which returns date.

3.UPDATE, DELETE, etc used to update/delete values in fields.

4. ALTER TABLE => Just as we can add, delete. modify values in rows, we can change columns for a given table. i.e add columns, remove columns using this cmd.

5. SHOW TABLES; # shows all tables in selected database.

6. SHOW COLUMNS from table_name; #This shows all the info for all columsn from the given table.

The 3 cmds, SHOW DATABSES, SHOW TABLES and SHOW COLUMNS can give all the info about whatever is present in any data on a sql server. Databases have tables, and tables have columns. Columns are the lowest hier that store the data. To prevent anyone from accessing all this info, mysql provides privileges at finer level up to the table. So, only privileged users can access a certain database and certain table, thus providng security to data stored. Database "mysql" stored by default in mysql server has one of the tables as "user". If we list columns of user, we will see these privileges.

ex: SHOW COLUMNS FROM user; -- Shows all colums as User, Host,  various privileges, etc (total of 45 columns) from table user in database named mysql.

+------------------------+-----------------------------------+------+-----+-----------------------+-------+
| Field                  | Type                              | Null | Key | Default               | Extra |
+------------------------+-----------------------------------+------+-----+-----------------------+-------+
| Host                   | char(60)                          | NO   | PRI |                       |       |
| User                   | char(32)                          | NO   | PRI |                       |       |
| Select_priv       | enum('N','Y')                   | NO   |         | N                   |       |

7. SHOW VARIABLES; => shows all variables and their values in use. variables are current database configuration settings (set by default or modified by user). It shows that default port for mysql server pgm running is 3306.

8.SELECT => This is the most used cmd to select fields from a given table. We can use a lot of complicated search criteria (by using WHERE clause) and get only the rows and columns meeting that search criteria.

ex: SELECT * FROM user; => This shows all fields (columns) for all rows from table "user" in database "mysql" (assuming mysql db was selected before). * denotes "all".
ex: SELECT name, salary FROM tutorials_tbl WHERE tutorial_author = 'Sanjay'; => HEre we select 2 fields (columns) from table "tutorials_tbl". WHERE clause adds extra filtering criteria, saying the field "tutorial_author" needs to be "Sanjay". So, only those rows will be selected which match the WHERE clause, and only 2 columns "name" and "salary" would be displayed. If * was used, all columns would be displayed. Many other keywords as AND, OR, <, !=, etc available for doing a match.

 3. Misc cmd => There are tons of other cmds for doing doing lot of other things with databases.

Adding new user to db:

So far we did everything on mysql db as "root" user. However, we'll want to allow valid users besides root to access mysql db. This can be done via creating a user, and then granting it privileges, Done via following cmds:

mysql > CREATE USER 'ashish'@'localhost' IDENTIFIED BY 'password'; #here, we created a new user 'ashish', but no privileges assigned, so we can't login with this user yet

mysql> GRANT ALL PRIVILEGES ON * . * TO 'ashish'@'localhost'; #grant all privileges to this new user when logged from localhost. First * refers to database while second * refers to  table. So, here we grant all privileges to user ashish on all db and all tables. Not a safe thing to do, as now user 'ashish' can modify anything in any db.                                          

mysql> FLUSH PRIVILEGES; #To reload all the privileges so they take effect immediately

 mysql> use mysql; #To check ifnew user has been added, we check in "mysql" database, and "user" table.

mysql> select * from user; # this shows a row for new user "ashish"

support for mysql in other languages:

We can connect to mysql server via mysql client by invoking it from terminal. However, the real value in mysql db is when we can connect to the db from within some other pgm. For this reason, popular web languages as php, python, perl, etc provide functions to connect to mysql server, rd/wrt to db, and basically do everything from within the language. php language provides extensive support for mysql. Details of this approach are explained in php tutorial.

Migration of Database: Copying/Moving MySQL Database from 1 server to other is not difficult. We know that database contents are stored in *.idb file (for InnoDB). However, moving these files by themselves is not going to work, as they rely on *.frm files and host of other files. Also, if the storage engine is different, then a conversion would be required from one format to other. To prevent users from all this hassle, mysql provides various utilities to copy/move the database.

The mysqldump program is used to copy or back up tables and databases. It can write the table output either as a Raw Datafile or as a set of INSERT statements that recreate the records in the table.

1. As raw datafile: ex:

> mysqldump -u root -p --no-create-info  --tab=/tmp/dir1 tutorials_DB tutorials_TABLE => this dumps table=tutorials_TABLE from database=tutorials_DB to file in /tmp/dir1 dir

2. in sql format: we can also dump selected tables or entire database in sql format in a separate file. Here, we write out output file as series of sql CREATE/INSERT cmds, that will recreate the original table.

> mysqldump -u root -p tutorials_DB tutorials_TABLE > dump.txt => same as above except that --tab option is not used. dump.txt is a regular sql file, that has "CREATE TABLE tutorials_TABLE" and "INSERT INTO tutorials_TABLE". To dump multiple tables from the database, provide name of each table separated by space (i.e tutorials_DB tutorials_TABLE1  tutorials_TABLE2 > dump.txt)

> mysqldump -u root -p TUTORIALS_DB > database_dump.txt  => this dumps entire database instead of tables

So, to move copy database from 1 server to other, use these 3 cmds below:

• mysqldump -u root -p MY_OLD_DB > database_dump.sql => dump all database in this file on old server from where you want to transfer
• mysqladmin -u root -p create MY_NEW_DB => create new database named MY_NEW_DB on new server where you want to transfer files to
• mysql -u root -p MY_NEW_DB < database_dump.sql =>Now run the dumped sql file on this new DB, which will create/insert all tables into this new db, using cmds from *.sql file.
  
   
  
   

Details

Published: Tuesday, 28 May 2019 13:45

Hits: 2004

iverilog - open source verilog simulator:

Many open source simulators available. We'll talk mainly about icarus as it's most widely used.

Verilator: commercial quality open source simulator. It's verilog to C translator. Translates Verilog/SV into C++/SystemC code resulting in very fast sim times. Verilator is 100X faster than interpreted verilog simulators as Icarus verilog. More info here: https://www.veripool.org/wiki/verilator

verilog2cpp: another open source simulator. More info here: http://verilog2cpp.sourceforge.net/

icarus verilog: (aka iverilog)

An open source verilog simulator written in C++. Very complicated, supports all verilog constructs. More info: http://iverilog.icarus.com/

Download from here: https://github.com/steveicarus/iverilog

On CentOS, we can download binaries directly, instead of building it. This is on CentOS 7.5.1804 release. Type below cmd on any terminal.
yum install iverilog => It may show some http 404 errors, but eventually will find some other mirrors. On entering "Y" after it asks for confirmation, following gets downloaded and installed. Downloading packages: iverilog-10_2-2.el7.x86_64.rpm           
Installed: iverilog.x86_64 0:10_2-2.el7 Both iverilog and vvp are installedwhich iverilog => shows path as "/usr/bin/iverilog"
which vvp => shows path as "/usr/bin/vvp"There is very good beginner's material here: https://iverilog.fandom.com/wiki/User_GuideLet's copy the hello.v code shown on link above, and run testcase on it: ( > below is the prompt)> iverilog -o hello hello.v
> vvp hello
Hello, World
We can similarly try "counter" example. cp counter.v and counter_tb.v and run below cmds: (finish cmd is needed to get out of simulatoriverilog -o my_design  counter_tb.v counter.v
[root@DESKTOP-G2APOII counter]# vvp my_design
At time                    0, value = xx (x)
At time                   17, value = 00 (0) ....
** VVP Stop(0) **
** Flushing output streams.
** Current simulation time is 168 ticks.
> finish
** Continue **
To get dumpfile from sim, we can add following code anywhere in tb file

initial
 begin
    $dumpfile("test.vcd");
    $dumpvars(0,test);
 end

Now on running cmds again, we will see test.vcd which can be loaded on any waveform viewer.

GTKWave: An open source waveform viewer. More info here: http://gtkwave.sourceforge.net/

Download following software before installing GTKWave.

1. gperf: GNU perfect hash function generator. Link here: https://www.gnu.org/software/gperf/

    download from ftp://ftp.gnu.org/pub/gnu/gperf/
    Download latest version (gperf-3.1.tar.gz). Extract and follow these steps:

    - cd gperf-3.1

    - ./configure

   - make

   - make check => optional. Runs test cases

   - sudo make install => installs it in /usr/local/bin/gperf

 2. gtk: It's used for creating GUI. Link here: https://www.gtk.org/.

  If you go to download page for Linux, you will see latest version as 3.24, while earlier gtk2 version as 2.24. I think you can download either one, however there are lot of dependencies as Glib, Pango, Gdk-Pixbuf, ATK and GObject-Introspection. I tried to install "gtk+-2.24.0", but it fails looking for these dependencies:

configure: error: Package requirements (glib-2.0 >= 2.27.3    atk >= 1.29.2    pango >= 1.20    cairo >= 1.6    gdk-pixbuf-2.0 >= 2.21.0) were not met:

No package 'glib-2.0' found
No package 'atk' found
No package 'pango' found
No package 'cairo' found
No package 'gdk-pixbuf-2.0' found

 I tried to install pango, cairo, glib. glib required meson, which could not be installed due to python 3.5 dependency (even though python3 was installed). Fortunately, CentOS repository has gtk pkg available for installation via yum, so I took that route:

 - yum groupinstall "Development Tools"

- sudo yum install gtk+-devel gtk2-devel => this will download and install gtk with all required dependencies

You will see these 2 pkg installed:

Installed:    gtk+-devel.x86_64 1:1.2.10-77.el7                              gtk2-devel.x86_64 0:2.24.31-1.el7        

Once you have downloaded above gperf and gtk package, we can download GTKWave (gtkwave-3.3.101.tar.gz) at bottom of gtkwave page. Extract it, and in the extracted dir, we'll see a README/INSTALL that lists the steps. Here's the steps:

1. cd gtkwave-3.3.101

2. ./configure --disable-xz => This will error out if the option is not used since xz is not installed.It's a compression software, we'll disable it for now. This is what you see on screen if xz not disabled:

configure: error: LZMA support for VZT is enabled, but xz could not be found.
Please install the xz-devel package, see the http://tukaani.org/xz website, or use the --disable-xz flag.

3. make

4. sudo make install => should get installed in /usr/local/bin/gtkwave

Now run "gtkwave test.vcd". It should bring up gtkwave gui.

NOTE: You get error like this while running gtkwave:

tkwave: error while loading shared libraries: libtcl8.7.so: cannot open shared object file: No such file or directory

This can happen when there are multiple versions of tcl/tk installed on your system. yum install or manual install of tcl/tk may not show any error, but when trying to run a software, we may see errors related to tcl/tk files. Best way is to remove tcl/tk, and start fresh from manual install of tcl/tk (yum may not fix the issue as "yum install" will always show everything uptodate and refuse to update). You may want to use below option to force a particular tcl/tk version to be used in step 2 above, and then continue with step 3 and 4:

./configure --disable-xz --with-tcl=/usr/lib64/tcl8.5 --with-tk=/usr/lib64/tk8.5 => replace version 8.5 with whatever version you have on your system. I used both 8.5 and 8.7. This did not work for me, so I had to reinstall tcl/tk 8.7. After that, it worked, without using these tcl/tk options.

 -----------------