dohashi

1197 Reputation

10 Badges

19 years, 204 days
I am a Senior Software Developer in the Kernel Group, working on the Maple language interpreter. I have been working at Maplesoft since 2001 on many aspects of the Kernel, however recently I have been focusing on enabling parallel programming in Maple. I have added various parallel programming tools to Maple, and have been trying to teaching parallel programming techniques to Maple programmers. I have a Master's degree in Mathematics (although really Computer Science) from the University of Waterloo. My research focused on Algorithm and Data Structure, Design and Analysis.

MaplePrimes Activity


These are Posts that have been published by dohashi

 

I don't really have anything prepared for today, so I'd like to ask you a few questions about the posts I've made so far.  My goal for this blog was to give typical Maple programmers the information they needed to start trying parallel programming. 

  1. My posts progressed fairly quickly, building up to the the Task Programming Model. Did I move too quickly?  Were there topics that I did not explain well enough or that you felt needed more explaination?
  2. As my goal was to present the Task Programming Model, I skipped a deeper explaination of Threads:-Create style of programming.    Would you like to know more about that type of low level threaded programming?
  3. Most of the examples I used were artifical ones that illustrated the points I was trying to make.  Would you have prefered real world examples instead?
  4. Did reading my posts get you to actually try writing a parallel algorithm?  If yes, did you succeed?  If no, why not?
  5. Was the formatting ok, especially the code?  Each post included an worksheet containing the examples from the post so I did not worry too much about ease of copy and paste.
  6. What else would you like to know about?  I am definitely planning a post on GPU computing, but since it is not really a Maple topic I delayed it till after I am finished with the Maple topics.

Any other feedback you would like to provide would also be appreciated, although I'd like to keep focused on the topics discussed in my blog, and less about Maple in general.

In my previous posts I have discussed various difficulties encountered when writing parallel algorithms. At the end of the last post I concluded that the best way to solve some of these problems is to introduce a higher level programming model. This blog post will discuss the Task Programming Model, the high level parallel programming model introduced in Maple 13.

In my previous posts I discussed the basic difference between parallel programming and single threaded programming. I also showed how controlling access to shared variables can be used to solve some of those problems. For this post, I am going to discuss more difficulties of writing good parallel algorithms.

Here are some definitions used in this post:

  • scale: the ability of a program to get faster as more cores are available
  • load balancing: how effectively work is distributed over the available cores
  • coarse grained parallelism: parallelizing routines at a high level
  • fine grained parallelism: parallelizing routines at a low level

Consider the following example

In the previous post, I described why parallel programming is hard. Now I am going to start describing techniques for writing parallel code that works correctly.

First some definitions.

  • thread safe: code that works correctly even when called in parallel.
  • critical section: an area of code that will not work correctly if run in parallel.
  • shared: a resource that can be accessed by more than one thread.
  • mutex: a programming tool that controls access to a section of code

In my previous post, I tried to convince you that going parallel is necessary for high performance applications. In this post, I am going to show what makes parallel programming different, and why that makes it harder.

Here are some definitions used in this post:

  • process: the operating system level representation of a running executable. Each process has memory that is protected from other processes.
  • thread: within a single process each thread represents an independent, concurrent path of execution. Threads within a process share memory.

We think of a function as a series of discrete steps. Lets say a function f, is composed of steps f1, f2, f3... e.g.

1 2 3 4 5 6 Page 5 of 6