Showing posts with label Complex numbers. Show all posts
Showing posts with label Complex numbers. Show all posts

28 March 2012

Amazing Abstract Algebra: Groups and Rotations


A little while ago I noticed a very interesting group, I am sure it is well known in math circles but I find it intriguing for a number of reasons and I have recently learned more about Abstract Algebra which ties in some more interesting ideas. The group relates to imaginary numbers, it is the set {1,-1,i,-i} which is closed under multiplication. It has an identity which is 1. Each element has an inverse, -1 is its own inverse: -1 · -1 = 1 and -i · i = 1. The following cayley table shows all this:


1 i -1 -i
1 1 i -1 -i
i i -1 -i 1
-1 -1 -i 1 i
-i -i 1 i -1

If you are new to these ideas I recommend looking at my previous post on abstract algebra, especially since we will be looking at some isomorphisms again. In order to talk about the ideas in the post we need to get some group theory concepts out of the way first, don't worry they aren't hard in fact they're fairly intuitive. Also in this post I am restricting the discussion to finite groups, groups with a finite number of elements. In fact the first idea is the concept of group order.

The order of a finite group is simply the number of elements of the set the group is defined on and is denoted as |G|.

The group above has an order of 4.

The next idea is that of exponentiation, which can be thought of as a subset of "classical" exponentiation on the field of real numbers, see identities here. Exponentiation on group elements extends to integer exponents only and refers to repeated application of the binary operation to the element itself. So:


an = a·a·a·a...a ( a "times" itself n times)

a-n = a-1·a-1·a-1·a-1...a-1 (a-1 "times" itself n times or the inverse of an)

a0 = e (where e is the identity element)

The laws of exponents are:

am an = am+n

(am)n = amn

a-n = (a-1)n = (an)-1


Just as a group has an order an element of a group also has an order and it is related to exponentiation and it is defined as:

If there exists a nonzero integer m such that am=e, then the order of the element is defined to be the least positive integer n such that an=e. Also if there does not exist any nonzero integer m such that am=e, we say that a has order infinity.

So the elements in our group (⋅, {1,-1,i,-i}) above have the following orders:

11 = 1 (this is the identity and has order 1) [11 = 1, ...]

-12 = 1 (has order 2) [-11 = -1, -12 = 1, ...]

i4 = 1 (has order 4) [i1 = i, i2 = -1, i3 = -i, i4 = 1, ...]

-i4 = 1 (has order 4) [-i1 = -i, -i2 = -1, -i3 = i, -i4 = 1, ...]

Order is also sometimes referred to as period and you should note that each of the above will cycle through each power listed in the square brackets so for both the order 4 elements {i, -i}: a1n=a1, a2n=a2, a3n=a3, a4n=a4 for all integers n>0 and the values will cycle through the list which would be the case for any element of finite order.

As it turns out elements like i and -i which cycle through all of the elements of a group exactly once in the period have an order that is equal to the order of the group. These define what is known as a cyclic group. So our group is cyclic and any element which cycles through all elements once is the generator so i and -i are generators for the group.

Another interesting result from group theory is that any cyclic group of order n is isomorphic to the group (+,Zn) which is the integers (mod n) under addition so our group above is just a renaming of Z4 see the similarity to the cayley table below:


+ 0 1 2 3
0 0 1 2 3
1 1 2 3 0
2 2 3 0 1
3 3 0 1 2

Now I want to take a look at another important group, it's called the dihedral group, this one is known as D4 and has the cayley table:

  R0 R1 R2 R3 V H D1 D2
R0 R0 R1 R2 R3 V H D1 D2
R1 R1 R2 R3 R0 D1 D2 H V
R2 R2 R3 R0 R1 H V D2 D1
R3 R3 R0 R1 R2 D2 D1 V H
V V D2 H D1 R0 R2 R3 R1
H H D1 V D2 R2 R0 R1 R3
D1 D1 V D2 H R1 R3 R0 R2
D2 D2 H D1 V R3 R1 R2 R0

The dihedral group is the group of symmetries of a regular polygon, including both rotations and reflections. In this case we will consider the rotations and reflections of a square denoted as D4 represented visually as four couterclockwise rotations of a square:



And the following four reflections, vertical, horizontal, and two diagonal reflections:




With each image is included the permutation, actually dihedral groups are permutation groups and D4 is a set of 8 permutations, hence the order of the group is 8. It is a subgroup of the group of all permutations of four items which is known as the Symmetric Group which has order 4!=24, it's denoted as S4. The interesting thing here is the rotations form a subgroup that is isomorphic to our group from above and Z4. In a sense this isn't really surprising since it is a cyclic group of order four meaning it cycles through 4 items just as a square cycles through four rotations.


As you can see from the color coding, the elements {R0, R1, R2, R3} form a subgroup, a subgroup is just a subset of the elements that is still a group. These subgroup elements forming the upper left hand corer of the table are color coded to show the isomorphism between this subgroup and Z4 and also our group (⋅, {1,-1,i,-i}). Also R0 is the identity which makes sense since it is a zero rotation, which means that you didn't rotate it, or you did by 360° aka one tau.


Our Z4 isomorphic group (⋅, {1,-1,i,-i}) also has a subgroup. Our original arrangement of the cayley table was ordered to show the Z4 isomorphism. To show our subgroup we need to rearrange the cayley table:


1 -1 i -i
1 1 -1 i -i
-1 -1 1 -i i
i i -i -1 1
-i -i i 1 -1

Now it should be fairly easy to the subgroup (⋅, {1,-1}) again forming the upper left corner in the table above, or shown by itself:


1 -1
1 1 -1
-1 -1 1

Now if you think about it our subgroup which is just another group is a group, 1 is the identity and -1 is its own inverse, actually it turns out that all subgroups of cyclic groups are also cyclic groups and since all cyclic groups are isomorphic to Zn groups, this group is actually one that we have seen before it's just Z2 addition aka exclusive-or, etc.

Also there are some remarkable connections to number theory in regards to the order of subgroups and the order of elements, for example the order of our subgroup is 2 which evenly divides 4 the order of the group, the order of a subgroup will always evenly divide the order of the group. Also the order of each element evenly divides the order of the group. I suspect that these ideas have profound implications.


Going back to our original group there are a couple of interesting things about imaginary numbers, first are the following equations:





If we consider the sequence of raising i to successive powers, multiplying by I each time, if we start with 1, we get the sequence (1, i, -1, -i), these are actually coordinates in the complex plane and can be written as (0i+1, 1i+0, 0i-1, -1i+0). This cycle (increasing powers of i) is the following (counter clockwise) rotation in the complex plane:



Conversely if we divide, or repeatedly multiply by -i, we get the following sequence (1,-i,-1,i) which correspond to the complex numbers (0i+1, -1i+0, 0i-1, 1i+0) and is the opposite (clockwise) rotation:



Another interesting seemingly related concept is the idea of turning number or winding number of a curve, the red circle with clockwise rotation corresponds to curve with a winding number of -1 and the blue curve with positive increasing powers of i has a winding number of 1.


As we saw our group is cyclic and isomorphic to Z4 and isomorphic to the rotation subgroup of the dihedral group D4 and both of the generators of our group correspond to rotation in the complex plane. Counterclockwise rotation corresponds to the period of i and clockwise rotation corresponds to the period of -i.

27 November 2011

Eleven Equations True Computer Science Geeks Should (at Least Pretend to) Know




This idea is a complete rip off an article that appeared in Wired a little while ago and it got me thinking what would my list for Computer Science look like?  Plus I thought it might be a fun post and unlike the Wired list this one goes to eleven.  So here they are in no particular order:



Binomial Coefficient



The Binomial Coefficient equation generates Pascal’s Triangle and gives you the coefficients for the Binomial Theorem these ideas are often attributed to Pascal but in fact they have been known in part for over a millennia.


As I mentioned that this list is no particular order and I don’t wish to play favorites but if there is one equation here that you should really consider learning and committing to memory this is it. It is central to Combinitorics which has connections to many areas of math, I guess I should qualify that if you are serious about computer science and math related programming topics then you should be able to recite and apply it from memory, even when you are drunk, sleep deprived or being held at gun point. Ok that might be a bit much but you get the idea.  If you are programmer and you haven’t learned it, or need a refresher, I have a post that relates it to the Java Collections API.



Demorgan’s Laws


Logical form:


Set Form:



Ok so that’s like four "equations" for DeMorgan’s Laws, one can’t help but to struck by the similarity between the two sets and this is not a coincidence these two sets of formulas are essentially the same thing, they are special cases of complemented distributive lattices  which means technically it’s really just two formulas:




In this case the ∨ symbol means lattice join operation and the ∧ symbol is the lattice meet operation and the dash with the right downward mark means lattice complementation, I used this to differentiate from the tilde for Boolean complementation.  Also these two formulas are categorical duals of each other, which means that if one were smart and had a good grasp of Category Theory we could probably be really OCD about it and get it down to one formula.


Understanding Set Theory and Boolean Algebra are very important basic concepts in our profession and the Boolean version can make you better programmer, as you can use it to refactor logical if statements


Eigenvector and Eigenvalue



This equation and these concepts are not only central to Linear Algebra but these Linear Algebraic ideas and others are both used and extend into many other areas of math including Linear Transformations and Graphs in terms of matrices like the Adjacency Matrix and much more.


Linear Algebra is becoming increasingly central to the types of things we are doing in our profession, it is used heavily in Statistics and Machine Learning not to mention the Page Rank Algorithm is based in part on this equation.



Pumping Lemma for Regular Languages



No Comp Sci list would be complete with at least one formula from either Formal Language Theory or Automata Theory.  The problem is that these two are pretty different from other areas of math in terms of "Equations" I tried to think of some basic Ideas and The Pumping Lemma came to mind, and I found the above formula concisely expressed on Wikipedia.  This version of the Pumping Lemma is a more limited version of Another Theory which is used to check whether a Language is Regular. Also this is maybe not the best choice as it is pretty dense, if you want a better explanation I recommend this.


While this "equation" is pretty specialized, it is actually a special case of the more general Iteration Theorem, the real point here is really about more general domain knowledge.  It is central to what you do every day you are programming such as compilers and regular expressions not to mention the almost ubiquitous Kleene Star.


Information Entropy


I confess after all my machinations to try to reduce Demorgan’s Laws down to less equations I am totally cheating here by pulling a twofer in including two formulas, both Shannon’s Information Theory:



And the formula for Chaitin’s Constant, which is associated with Algorithmic Information Theory and Kolmogorov Complexity:



Interestingly this area has a parallel in the physical word, in terms of Thermodynamic Entropy, which also parallels the Boltzman Equation mentioned in the Wired Article.  Seth Loyd draws some interesting connections between computation and the physical world including Entropy and Boltzman’s Equation.


In programming and computer science information is our business and these ideas are central to many things that we deal with especially the storage, transmission, computational transformation (like compression), and computation of information.  We’ve already seen the possible relationship between both of these theories and Reusability and Software Frameworks.



Bayes' Theorem



Of all of equations on the list, Bayes' Theorem may stand out on its own merits as being one of the most recognizable and it might even qualify as a poster child for the statistical revolution sweeping our industry.  It rose from obscurity to being used for spam detection and with such applications as classifiers, Machine Learning, Data Mining and more.


Mathematically it is part of a rift in statistics and probability and I confess I may not yet call myself "an initiate of the bayesian conspiracy" but it hard to deny its utility plus there seems to be a more axiomatic basis that relates to Boolean Logic Set Theory, which makes it all the more alluring.



Fermat’s Little Theorem




These two equations of Fermat’s Little Theorem are really the same equation written in two different forms, where (a ⊥ p) means coprime and (p ∈ P) means prime.


This is a beautiful little theorem in Number Theory which can be generalized and written even more succinctly using Eulers Totient Function which is represented by φ (n):



These ideas are crucial for understanding encryption algorithms like RSA and there’s this cool result.


Natural Join



Natural Join, like all of the Relational Algebra Join Operations is a composition of the more basic, operations: Selection (σ), Projection (π), Cartesian Product (×), Set Difference (-) and Union (∪).  Natural Join is the Cartesian Product of two tables selecting matching rows and eliminating duplicate columns.  (In the formula above the projection (π) is the set (A) of attributes with duplicates removed and then selects (σ) the set (C) of common attributes which match in both sets and are equal in both sets.)


The Relational Algebra is in my opinion a bit of an odd beast when it comes to algebraic structures, but if you use SQL you are already using an implementation of it.  As data persistence becomes more heterogeneous I think understanding these concepts, will become more important.  Additionally Relational Algebra also has strong ties to Abstract Algebra[ and Category Theory[.



The Fixed-Point (Y) Combinator



Just as we need a "equation" from Formal Language Theory or Automata Theory we really should have one that is related to Lambda Calculus, in this case this equation is in the untyped lambda calculus.  Even though the Fixed-point Combinator stands up on its own, it is fairly well known, well at least in name due to the use of it by Paul Graham for his incubator (Y-Combinator).


The Fixed-Point Combinator allows one to implement anonymous recursion which is a powerful programming technique.  It also ties into some deeper theoretical aspects of computer science and logic such as Combinatory Logic.



O(n)


Many CS majors may cringe at this, and another possible reaction is that’s not much of an equation.  There’s actually quite a bit to this for example did you know there are some substantial and quite beautiful equations associated with this seemingly simple formula.  They are:




Also the following is a definition of (lim sup) Limit Superior:



Where inf is infimum and sup is supremum two concepts that seem to show up quite a bit especially when you are concerned with bounds like with Lattices and they also seem to show up in relation to Topology.


And we all know why we should care about his one: Algorithms, Algorithms, Algorithms.



Euler’s Identity



Euler’s Identity is also listed in Wired’s article and I have to agree that it is one the most beautiful and intriguing formulas in math.


Now this isn’t really as relevant to CS as the other formulas but if you a true computer geek you are hopefully some kind of math geek, and this is such a great formula, not to mention that it appears in The Simpsons' Homer3. It’s actually a specific case (where θ = π) of Euler’s Formula:



These formulas will give one a better understanding complex numbers which are needed in other areas of advanced math also these formulas relate to Fourier Analysis which does have applications in Computer Science.


I am curious to see how well this list holds up, how will I feel about it in a year?  Also in writing this I realized that I still have quite a bit more to learn about at least half of this list, if not all of it, which made it hard to write as it was quite a little (unfinished) research project. I have already written individual articles on several of these equations and have at least two in the works for Linear Algebra, at least two on the Relational Algebra and one planned for Pascal’s Triangle, and those are just the ideas I’ve already had, who knows what the future holds.  I hope you enjoyed it as much as I did writing it.


The Set version may not apply to all cases in set theory, this assumes Union and Intersection over the Power Set of a Set Universe, this would be closed under these operations and would include complements of all elements.  The Set Universe is the upper bound and the empty set is the lower bound.