Your definition of the function

a deinition of binom so that m so that binom(n,k) returns  n k  for all n greater than 0, and k greater than or equal to 0 using the definition of the binomial above

is nonsense because n > 0 and k >= 0 are not mutally exclusive conditions.

Which of the two results do you consider to be simplified "correctly"?

If you're working with complex expressions, and you want to assume that all variables are real, you should use evalc.

To get more help, you'll need to post the full code.

@H-R Let a be a negative real and r a rational number. In order for the function a^r to be continuous as a complex-valued function of r, it can't be allowed to switch from negative to imaginary every time the denominator of r switches from odd to even. Instead, the relation x^y = exp(y*ln(x)) (which holds for any complex x and y other than x=0) is used to determine the value, using the principal value of the logarithm.

(-1)^(1/3) = exp(ln(-1)/3) = exp(I*Pi/3) = cos(Pi/3) + I*sin(Pi/3) = 1/2 + I*sqrt(3)/2 = (1+I*sqrt(3))/2.

If you expand the cube of this complex number, using purely the means of high-school algebra, you'll see that you get -1:

((1+I*sqrt(3))/2)^3 = (1+3*I*sqrt(3)+3*I^2*3+I^3*3*sqrt(3))/8

= (1+3*sqrt(3)*I-9+3*sqrt(3)*I)/8 = (1-9)/8 = -1.

Maple is designed to work with complex numbers by default. So it's important that the primary operations like exponentiation work correctly for complex numbers. So, the less-important real-number case is handled by a secondary function, surd.

@vv This can be made more automatic by using IntegrationTools:-Change:

G:= (k::integer, n::integer)->
     (value@expand)(
          IntegrationTools:-Change(
               Int((1-cos(k*x)*cos(n*y))/(sin(x/2)^2+sin(y/2)^2)/2/Pi^2, [x,y]=~ 0..Pi),
               {x,y}=~ 2*~arctan~({u,v})
          )
     )
:

@Rouben Rostamian  You wrote:

In fact, I would appreciate it if someone points me to a reference.

See https://en.wikipedia.org/wiki/Moore%E2%80%93Penrose_pseudoinverse

In particular, see the small section on projectors.

@Preben Alsholm The builtin frem performs a similar function. It's more analogous to mods than to modp.

So what happens when you click "+ Manage Tags"?

@marians I just restored your original Question as best that I could. You should review it to make sure that you approve. You can edit by pulling down the "More" menu, which is the last thing on the right at the bottom of any post. I am sorry that I was harsh with you about removing the Question. Apparently it was an accident, and you had nothing to do with it. There've been problems in the past with people intentionally removing their Questions after they've been Answered.

@marians It appears that you edited the original Question and removed it. Please put it back! It's unacceptable to remove a Question after it's been Answered.

Regarding your latest question, I guess that you want to compute the conjugate using each pair of elements of A and make a new set with the conjugates? Is that correct? And you want to include the cases where i = j (which, of course, just reduces to the set A itself), right?

@acer Ah, I see that it doesn't work for ISOSURFACEs. Thanks for bringing this to my attention.

To correct this, I could search through the 4-d Array for the points where the fourth coordinate is "close to" 0. Then, among those points, search for the extremal values of the third coordinate, which is the z-coordinate. Would that work? The question remains How "close to" 0? What criterion is used by the plot renderer to decide whether to plot a point of an ISOSURFACE? 

@software_c It's common and normal for Maple to re-arrange an expression into a different term order than that with which it was input. Don't use that alone as an indicator that you've encountered a bug.

@Markiyan Hirnyk Do you dispute the cited quotation? Can you present any evidence that it isn't true? Can you cite any source that even casts a shadow of doubt on it? Any hint of doubt on the truth of the cited statement would cause major unrest in the world of cryptography and would definitely make the newpapers and broadcast news. As I said, the statement isn't a proven mathematical fact; it just represents the currently known state of affairs. This is a very, very active area of research.

@Markiyan Hirnyk The facts that I stated are very well-known results of computational number theory---so well known that they are sometimes mentioned in common newspapers. See the Wikipedia article Primality test. Some "algorithms" commonly used for primality testing haven't been proven correct. This includes the one used by Maple's isprime (see ?isprime). It's been proven that there's a theoretically fast (O(ln(n)^12)) primality testing true algorithm; however, I know of no practical implementation of this algorithm.

I invite you to redo my examples with randomly chosen primes.

@dorna01 Since fsolve found a root at

you might as well use something near that as an initial point. I also suggest that you set Digits to 15, but I have no evidence that that'll make any difference in this case. It's just generally a good value for Digits, balancing speed and accuracy.

@vv The plot

plot([Re,Im](DETM2), omega= -1e9..1e9)

leads me to believe that DETM2 is a perfectly reasonable nonconstant function.

There's a syntax error in your construction of the matrix N: An operator is required after the first 20.