Tag Archives: bayesian

text | wondering, The field of AI

The Field of AI (Part 02-6): A “Pre-History” & a Foundational Context

post version: 2 (April 28, 2020)
It has active components that help supply blood to penile organ. sildenafil 100mg tab Source When you read such ads again, tell yourself that it is not really that bad and you can learn to super cialis cheap live with it. This method goes well with their hectic timetable after and during school classes, while parents truly admire distance education the best viagra training. This herb is also known to enhance heart health in men, reduce oxygen damage to LDL or bad cholesterol in men, and protect them cialis 40 mg from the inside.

URLs for A “Pre-History” & a Foundational Context:

  • This post is the main post on a Pre-History & a Foundational context of the Field of AI. In this post a narrative is constructed surrounding the “Pre-History”. It links with the following posts:
  • The post here is a first and very short linking with on Literature, Mythology & Arts as one of the foundational contexts of the Field of AI
  • The second part in the contextualization is the post touching on a few attributes from Philosophy, Psychology and Linguistics
  • Following, one can read about very few attributes picked up on from Control Theory as contextualizing to the Field of AI
  • Cognitive Science is the fourth field that is mapped with the Field of AI.
  • Mathematics & Statistics is in this writing the sixth area associated as a context to the Field of AI
  • Other fields contextualizing the Field of AI are being considered (e.g. Data Science & Statistics, Economy, Engineering fields)


05 — The Field of AI: A Foundational Context: Mathematics & Statistics

Mathematics & Statistics

The word ‘mathematics’ comes from Ancient Greek and means as much as “fond of learning, study or knowledge”. Dr. Hardy, G.H. (1877 – 1947), a famous mathematician, defined mathematics as the study and the making of patterns[1]. At least intuitively, as seen from these different perspectives, this might make a link between the fields of Cognitive Science, AI and mathematics a bit more obvious or exciting to some.

Looking at these two simple identifiers of math, one might come to appreciate math in itself even more but, also one might think slightly differently of  “pattern recognition” in the field of “Artificial Intelligence” and its sub-study of  “Machine Learning.”[2] Following, one might wonder whether mathematics perhaps lies at the foundation of machine or other learning.

Mathematics[3] and its many areas are covering formal proof, algorithms, computation and computational thinking, abstraction, probability, decidability, and so on. Many introductory K-16 resources are freely accessible on various mathematical topics[4] such as statistics.[5]

Statistics, as a sub-field or branch of mathematics, is the academic area focused on data and their collection, analysis (e.g. preparation, interpretation, organization, comparison, etc.), and visualization (or other forms of presentation). The field studies models based on these processes imposed onto data. Some practitioners argue that Statistics stands separately from mathematics.

These following areas of study in mathematics (and more) lie at the foundation of Machine Learning (ML).[6] Yet, it should be noted, one never stops learning mathematics for specialized ML applications:

  • (Bayesian) Statistics[7]
    • Statistics.[8]
    • See a future post for more perceptions on probability
    • Probability[9] Theory[10] which, is applied to make assumptions of a likelihood in the given data (Bayes’ Theorem, distributions, MLE, regression, inference, …);[11]
    • Markov[12] Chains[13] which model probability[14] in processes that are possibly changing from one state into another (and back) based on the present state (and not past states).[15]
    • Linear Algebra[16] which, is used to describe parameters and build algorithm and Neural Network structures;
      • Algebra for K-16[17]. Again, over-simplified, algebra is a major part of mathematics studying the manipulation of mathematical symbols with the use of letters, such as to make equations and more.
      • Vectors[18]            
      • Matrix Algebras[19]
    •  (Multivariate or multivariable) Calculus[20] which, is used to develop and improve learning-related attributes in Machine Learning.
      • Pre-Calculus & Calculus[21]: oversimplified, one can state that this is the mathematical study of change and thus also motion.[22] Note, just perhaps it might be advisable to consider first laying some foundations of (linear) algebra, geometry and trigonometry before calculus.
      • Multivariate (Multivariable) Calculus: instead of only dealing with one variable, here one focuses on calculus with many variables. Note, this seems not commonly covered within high school settings, ignoring the relatively few exceptional high school students who do study it.[23]
        • Vector[24] Calculus (i.e. Gradient, Divergence, Curl) and vector algebra:[25] of use in understanding the mathematics behind the Backpropagation Algorithm, used in present-day artificial neural networks, as part of research in Machine Learning or Deep Learning and the supervised learning technique.
      • Mathematical Series and Convergence, numerical methods for Analysis
    • Set Theory[26] or Type Theory: the latter is similar to the former except that the latter eliminates some paradoxes found in Set Theory.
    • Basics of (Numerical) Optimization[27] (Linear / Quadratic)[28]
    • Other: discrete mathematics (e.g. proof, algorithms, set theory, graph theory), information theory, optimization, numerical and functional analysis, topology, combinatorics, computational geometry, complexity theory, mathematical modeling, …
    • Additional: Stochastic Models and Time Series Analysis; Differential Equations; Fourier’s and Wavelengths; Random Fields;
    • Even More advanced: PDEs; Stochastic Differential Equations and Solutions; PCA; Dirichlet Processes; Uncertainty Quantification (Polynomial Chaos, Projections on vector space)
Mini Project #___ : 
Markov Chains 
Can you rework this Python project by Ms. Linsey Bieda, to use Chinese or another language’s word list?
Project context: https://rarlindseysmash.com/posts/2009-11-21-making-sense-and-nonsense-of-markov-chains 
Code source: https://gist.github.com/3928224

[1] Hardy. H.R. & Snow, C.P. (1941).  A Mathematician’s Apology. London: Cambridge University Press

[2] More on “pattern recognition” in the field of “Artificial Intelligence” and its sub-study of  “Machine Learning” will follow elsewhere in future posts.

[3] Courant, R. et al. (1996). What Is Mathematics? An Elementary Approach to Ideas and Methods. USA: Oxford University Press  

[4] For instance (in alphabetical order):

[5] Meery, B. (2009). Probability and Statistics (Basic). FlexBook.  Online: CK-12 Foundation. Retrieved on March 31, 2020 from  http://cafreetextbooks.ck12.org/math/CK12_Prob_Stat_Basic.pdf

[6] a sub-field in the field of Artificial Intelligence research and development (more details later in a future post). A resource covering mathematics for Machine learning can be found here:

Deisenroth, M. P. et al. (2020). Mathematics for Machine Learning. Online: Cambridge University Press. Retrieved on April 28, 2020 from https://mml-book.github.io/book/mml-book.pdf AND https://github.com/mml-book/mml-book.github.io

Orland, P. (2020). Math for Programmers. Online: Manning Publications. Retrieved on April 28, 2020 from https://www.manning.com/books/math-for-programmers 

[7] Downey, A.B. (?).Think Stats. Exploratory Data Analysis in Python. Version 2.0.38 Online: Needham, MA: Green Tea Press. Retrieved on March 9, 2020 from http://greenteapress.com/thinkstats2/thinkstats2.pdf

[8] A basic High School introduction to Statistics (and on mathematics) can be freely found at Khan Academy. Retrieved on March 31, 2020 from https://www.khanacademy.org/math/probability

[9] Grinstead, C. M.; Snell, J. L. (1997). Introduction to Probability. USA: American Mathematical Society (AMS). Online: Dartmouth College. Retrieved on March 31, 2020 from https://www.dartmouth.edu/~chance/teaching_aids/books_articles/probability_book/amsbook.mac.pdf AND solutions to the exercises retrieved from http://mathsdemo.cf.ac.uk/maths/resources/Probability_Answers.pdf

[10] Such as: Distributions, Expectations, Variance, Covariance, Random Variables, …

[11] Doyle, P. G. (2006). Grinstead and Snell’s Introduction to Probability. The CHANCE Project. Online: Dartmouth retrieved on March 31, 2020 from https://math.dartmouth.edu/~prob/prob/prob.pdf

[12] Norris, J. (1997). Markov Chains (Cambridge Series in Statistical and Probabilistic Mathematics). Cambridge: Cambridge University Press. Information retrieved on March 31, 2020 from https://www.cambridge.org/core/books/markov-chains/A3F966B10633A32C8F06F37158031739  AND http://www.statslab.cam.ac.uk/~james/Markov/  AND  http://www.statslab.cam.ac.uk/~rrw1/markov/    http://www.statslab.cam.ac.uk/~rrw1/markov/M.pdf AND https://books.google.com.hk/books/about/Markov_Chains.html?id=qM65VRmOJZAC&redir_esc=y

[13] Markov, A. A. (January 23, 1913). An Example of Statistical Investigation of the Text Eugene Onegin Concerning the Connection of Samples in Chains. Lecture at the physical-mathematical faculty, Royal Academy of Sciences, St. Petersburg, Russia. In (2006, 2007). Science in Context 19(4), 591-600. UK: Cambridge University Press. Information retrieved on March 31, 2020 from https://www.cambridge.org/core/journals/science-in-context/article/an-example-of-statistical-investigation-of-the-text-eugene-onegin-concerning-the-connection-of-samples-in-chains/EA1E005FA0BC4522399A4E9DA0304862

[14] Doyle, P. G. (2006). Grinstead and Snell’s Introduction to Probability. Chapter 11, Markov Chains. Dartmouth retrieved on March 31, 2020 from https://www.dartmouth.edu/~chance/teaching_aids/books_articles/probability_book/Chapter11.pdf

[15] A fun and fantasy-rich introduction to Markov Chains: Bieda, L. (2009). Making Sense and Nonsense of Markov Chains. Online, retrieved on March 31, 2020 from https://rarlindseysmash.com/posts/2009-11-21-making-sense-and-nonsense-of-markov-chains AND https://gist.github.com/LindseyB/3928224

[16] Such as: Scalars, Vectors, Matrices, Tensors….

See:

Lang, S. (2002). Algebra. Springer AND

Strang, G. (2016). Introduction to Linear Algebra. (Fifth Edition). Cambridge MA, USA: Wellesley-Cambridge & The MIT Press. Information retrieved on April 24, 2020 from https://math.mit.edu/~gs/linearalgebra/ AND https://math.mit.edu/~gs/AND

Strang, G. (Fall 1999). Linear Algebra. Video Lectures (MIT OpenCourseWare). Online: MIT Center for Advanced Educational Services. Retrieved on March 9, 2020 from https://ocw.mit.edu/courses/mathematics/18-06-linear-algebra-spring-2010/video-lectures/ AND

Hefferon, J. Linear Algebra. http://joshua.smcvt.edu/linearalgebra/book.pdf  AND http://joshua.smcvt.edu/linearalgebra/#current_version  (teaching slides, answers to exercises, etc.)

[17] Algebra basics and beyond can be studied via these resources retrieved on March 31, 2020 from https://www.ck12.org/fbbrowse/list?Grade=All%20Grades&Language=All%20Languages&Subject=Algebra

[18] Roche, J. (2003). Introducing Vectors. Online Retrieved on April 9, 2020 from http://www.marco-learningsystems.com/pages/roche/introvectors.htm

[19] Petersen, K.B & Pedersen, M.S. (November 15, 2012). The Matrix Cookbook. Online Retrieved from http://matrixcookbook.com and https://www2.imm.dtu.dk/pubdb/views/edoc_download.php/3274/pdf/imm3274.pdf

[20] Such as: Derivatives, Integrals, limits, Gradients, Differential Operators, Optimization. …See a leading text book for more details: Goodfellow, I. et al. (2017). Deep Learning. Cambridge, MA: MIT Press + online via www.deeplearningbook.org and its https://www.deeplearningbook.org/contents/linear_algebra.html Retrieved on March 2, 2020.

[21] Spong, M. et al. (20-19). CK-12 Precalculus Concepts 2.0. Online: CK-12 Retrieved on March 31, 2020 from https://flexbooks.ck12.org/cbook/ck-12-precalculus-concepts-2.0/ and more at https://www.ck12.org/fbbrowse/list/?Subject=Calculus&Language=All%20Languages&Grade=All%20Grades

[22] Jerison, D. (2006, 2010). 18.01 SC Single Variable Calculus. Fall 2010. Massachusetts Institute of Technology: MIT OpenCourseWare, https://ocw.mit.edu. License: Creative Commons BY-NC-SA. Retrieved on March 31, 2020 from https://ocw.mit.edu/courses/mathematics/18-01sc-single-variable-calculus-fall-2010/#

[23] A couple of anecdotal examples can be browsed here: https://talk.collegeconfidential.com/high-school-life/1607668-how-many-people-actually-take-multivariable-calc-in-high-school-p2.html and https://www.forbes.com/sites/johnewing/2020/02/15/should-i-take-calculus-in-high-school/#7360ae8a7625 .  In this latter article references to formal studies are provided; it is suggested to be cautious about taking Calculus, let alone the multivariable type. An online course on Multivariable Calculus for High school students is offered at John Hopkins’s Center for Talented Youth: Retrieved on March 31, 2020 from https://cty.jhu.edu/online/courses/mathematics/multivariable_calculus.html Alternatively, the MIT Open Courseware option is also available: https://ocw.mit.edu/courses/mathematics/18-02sc-multivariable-calculus-fall-2010/Syllabus/

[24] Enjoy mesmerizing play with vectors here: https://anvaka.github.io/fieldplay  

[25] Hubbard, J. H. et al. (2009). Vector Calculus, Linear Algebra, and Differential Forms A Unified Approach. Matrix Editions

[26] The study of collections of distinct objects or elements. The elements can be any kind of object (number or other)

[27] Boyd, S & Vandenberghe, L. (2009). Convex Optimization. Online: Cambridge University Press. Retrieved on March 9, 2020 from https://web.stanford.edu/~boyd/cvxbook/bv_cvxbook.pdf

[28] Luke, S. (October 2015). Essentials of Metaheuristics. Online Version 2.2. Online: George Mason University. Retrieved on March 9, 2020 from https://cs.gmu.edu/~sean/book/metaheuristics/Essentials.pdf