@fairuzalwani 

Okay, I understand more now. Please let me know if this MEC procedure does what you need:

MEC:= proc(N::nonnegint)
local z, ic, Escape, Capture, MinPts, Used;
     for ic from 0 to N do
          if assigned(Used[ic]) then next end if;
          MinPts[ic]:= true;
          z:= ic;
          while F(z) <= N do
               z:= F(z);
               Used[z]:= true;
               Escape[z]:= true
          end do;
          z:= ic;
          while Finv(z) <= N do
               z:= Finv(z);
               Used[z]:= true;
               Capture[z]:= true
          end do
     end do;
     (`{}`@indices)~([MinPts, Escape, Capture], 'nolist')
end proc:

This is very similar to your original code. Instead of the IsEmpty that you used to mark the points that haven't been used, I use Used to mark the points that have been used. And, of course, these are all Maple tables, not arrays; note the use of the built-in assigned in the if statement. All the Boolean infomation is conveyed by the fact that the element is or isn't assigned; the true that I used is just a dummy.

One thing that I still don't understand is why you think that you need to go to 2*N rather than N.

Here's a better, simpler Orbit procedure:

Orbit:= proc(Map::appliable, ic::nonnegint, Nit::nonnegint)
local orbit:= Array(0..Nit, [ic]), k;
     for k to Nit do orbit[k]:= Map(orbit[k-1]) end do;
     convert(orbit, list)
end proc:

Note that I use Array (with a capital A), rather than array. The use of array is deprecated, meaning that you should never use it. It only exists to support legacy code.

@Carl Love

By running the program MEC(200), I was able to quickly come up with a complete characterization of the minimum points, the escapees, and the capturees:

1. A nonnegative integer n is a minimum point (i.e., it's the minimal value in its F-orbit) iff it's of the form k*(k-1)/2 for some positive integer k. (These are also called the triangle numbers.)
2. If k is odd, then all the numbers between n and the next triangle number are escapees (i.e., they're on the increasing part of their orbit).
3. If k is even, then all the numbers between n and the next triangle number are capturees (i.e., they're on the decreasing part of their orbit).

(All of the above only applies to the case a = 4, b = 1. I haven't tried other pairs for a and b.)

(These are just conjectures from experimental evidence; they remain to be proven.)

Is this what you were trying to discover?

@Kitonum He's asking if there's any reason to prefer one form over the other given that he doesn't intend to apply f or its derivative to any specific value of t, which doesn't seem to be covered by your Answer.

@Markiyan Hirnyk Do you know that in English the phrase "so-called" is an insult, and that it's hyphenated? I don't understand your comment about capital letters; the OP hasn't misused such.

@pilpilet The body of your Reply is missing. Also note that I added an answer to your question about being sure of the accuracy without using intuition.

@fairuzalwani 

I'm sorry, but your attempt to describe the working of your program is mostly gibberish and nonsense to my English-reading mind. I was hoping that instead you could describe mathematically what you're trying to do, without reference to specific computer code. I understand that you expect that your procedure will form a partition of the integers from 0 to N.

At no point in the code do you ever assign false to a Boolean variable. That means that all Boolean variables are always true. That's obviously incorrect.

Your procedure makes no sense to me. For one thing, you never assign anything false, so they all must be true, right? How do you suppose that the Boolean arrays are initialized? It looks like you're trying to initialize them to true, right?

You should provide the code for F and Finv, a clear definition for the category MinPts, and why you think that there's no intersection of the categories Escape and Capture.

@perr7 You asked:

My original question was if the black circle in the xy-plane can be visible everywhere, also where the 3D probability density hides the circle. Do you know if that is possible?

Other than the options mentioned by Acer (such as transparency), I think that it may be possible in Maple's Classic GUI interface. If I recall correctly (it's been many years since I've used it), there was a nuisance feature such that lines always appeared on top of surfaces. I think that's exactly what you want in this case.

When the time "t" in the animation is increased, one needs to increase the number of of frames as well. Then it starts to take some time for Maple to do the calculations. It just takes long time, or should the code be written differently?

Changing 'sign(-x)' to signum(-x) will substantially reduce the time taken by each plot3d command because sign prevents them from running in the high-speed evalhf numeric environment. The command sign is strictly for symbolic polynomial algebra, and it has no place in numeric computation. 

My final goal is to make a short video, maybe 10 sec, of the animation. My strategy is to export the file as a GIF-file and then a find a online converter that converts the GIF-file to a video-file. Is that the right way to do it?

No, don't convert it to a video file---that'll substantially reduce the quality. The animation will play fine as a GIF file.

@sunflower 

No, they're not the same. The first one has an extra space. Look at them lined up:

printf ("*");
printf("*");

See? The first one is longer because it has an extra space. That extra space is illegal in Maple's 2D input.

@perr7 

The message "Length of output exceeds limit if 1000000" is NOT an error message. It's an indication that the computation has been sucessfully completed but the results are too long to show on screen. When you assign a plot or an animation to a variable, there's usually no need to see the algebraic form of the plot structure. Just end the command with a colon to suppress the output.

If you need further help, you'll probably need to post the full code so that we can execute it.

@Christopher2222 

Your procedure needs adjustment for the cases when the exponent (the second operand of the float) is nonnegative and for the special weird case of negative zero (which is a separate float in Maple). Here's my procedure that handles those cases:

`convert/floatsymbol`:= (a::coerce(float, (a::realcons)-> evalf(a)))->
     (e->
          nprintf(
               cat(
                   `if`(CopySign(1,a) < 0, "-", ""),
                   "%.",
                   max(-e,0),
                   "f",
                   `if`(e < 0, "", ".")
               ),
               abs(a)
          )
     )(op(2,a))
:

Usage:

convert(.006, floatsymbol);
convert(-1000., floatsymbol);
convert(-0., floatsymbol);

@Athar Shahabinejad 

The number that you got means the same as 5.275440947*10^6 or 5275440.947 or 5.275440947E6. The number format is called scientific notation.

@ 

In my experience, when it's applied to univariate polynomials, fsolve is totally reliable and very easy to use. Do you have an example where it's not?

@J4James The purpose of the seq is to specify which contours to plot. In this case, there are 21 evenly spaced from 1 to 3.

@Preben Alsholm 

I find the 14 + (-1) particularly odd. That should be covered by automatic simplification.

It's also odd that (x^2-1) got changed to (x^2+(-1)).

In some ways it seems as if -1 is being treated as a symbol.