Using Maple 2017 for 64bit Linux I get a smoothly rendered curve for this example, displayed inline in the Standard GUI, simply by removing the high numpoints option.

More particularly, for this example I get a jagged display for the inlined plot display if the value for the numpoints option is 2000 or greater, and a much smoother curve if numpoints is less than 2000 (or absent).

@_Maxim_ 

Loss of detail occurs due to scaling (and possibly by encoding which involves GUI-side conversion to PNG).

In the attachment below those details are still quite clear, by Preview2D without altering the size.

As mentioned, a benefit is that the point-probe manipulator can be used. Other benefits include the fact that it is regular output like a plot and so doesn't have the Embed restriction of one result per execution-group (which affects all programmaticly embedded component assemblies). Also, the GUI can be slower to render image backgroups on 2-D plots. I agree, if those benefits aren't needed then Embed serves better.

[edit] By supplying the extra option axes=none the Preview2D result can be a closer match to the image size, since the axes and tickmarks don't contributes to the sizing bookkeeping. But the point-probe details can still be enabled and accessed. (See new attachment) Perhaps a better default would be with axes=none.

2dpreview3.mw

@vv Nice example.

Testzero:=u->evalb(radnormal(expand(convert(u,exp)))=0):

Testzero( sin((3/7)*Pi)-sin((1/7)*Pi)+sin((2/7)*Pi)-sqrt(7)/2 );

                           true

That Testzero handles the original Matrix M. I hope nobody interprets this as meaning that I think finding a strong-enough but fast-enough zero detection mechanism is easy or even always possible.

[edited] Also, getting an effective Testzero won't help with expression swell and obtaining results that are unwieldy. So obtaining a Normalizer that handles the swell usefully while not taking a great deal of time can also be important. I realize that most of the participants in this thread are aware of the difficulties.

@Robert Israel I believe I observed that kind of thing for M[1..4,1..4] with Testzero=testeq.

@_Maxim_ The "Point Probe" manipulator is not available for 3-D plots. And ImageTools:-Preview displays a 3-D plot with an appropriate orientation and the color/shading applied to a plane where z=constant. (You can even rotate it with the mouse...)

The ImageTools:-Preview command was written before the background option for 2-D plots was introduced.

Here's a procedure which can be used to display an Image (Array or Matrix) in a 2-D plot, after which the manipulator can be used see coordinates of points in the image. You may have to use the right-click contextmenu item on the plot output to enable or set the manipulator, as per usual.

Preview2D:=proc(im::{Matrix,Array}(datatype=float[8]),
                {size::{[posint,posint],
                        identical(NoUserValue)}:=':-NoUserValue'},
                {scaleopts::list:=[]})
  # `scaleopts` get passed to ImageTools:-Scale while
  # any other passed arguments get passed to `plot`.
  local dims,wlo,whi,hlo,hhi,scaled,sizeopt;
  dims:=[rtable_dims(im)];
  if numelems(dims)<>2 then
     error "expecting an Array with 2 dimensions, got %1",
           numelems(dims);
  end if;
  wlo,whi := (lhs,rhs)(dims[2]);
  hlo,hhi := (lhs,rhs)(dims[1]);
  if size=':-NoUserValue'
    or ( size[1]=whi-wlo+1 and size[2]=hhi-hlo+1 ) then
     scaled := im;
     sizeopt := NULL:
  else
     scaled := ImageTools:-Scale(im, 1..size[2], 1..size[1],
                                 op(scaleopts));
     sizeopt := ':-size'=size;
  end if;
  plot('axes'="frame",'view'=[wlo..whi,hlo..hhi],
       'background'=scaled, sizeopt,
       'axis'=[':-thickness'=0,':-location'="low"], _rest);
end proc:

And here is is with your code. 2dpreview.mw

As for the need to wrap ImageTools command in try..catch to avoid overlong error messages, that appears to be a regression that occurred between Maple 2015.2 and Maple 2016.0. In Maple 2015 the error string generated by a procedure's param_processing was elided for large rtables. I have submitted a bug report.

@Christopher2222 That's what used to happen, as I stated above.

I haven't figured out yet whether unprotect/rewrite of `error` could work around the issue nicely enough. (Let alone whether examination of the callstack could make it appear as if the rethrown error were being emitted from the original location...)

@uomcsg Tom is suggesting that you would use the combine, rather than use trigsubs.

You can pass the optional argument trig to the combine command, to restrict the action to trig subexpressions.

Download combine_trig.mw

In my opinion the applyrule command is very weakly coded. I am not aware of any involved maple procedure or package that relies on it in a major way while not inheriting applyrule's many flaws.

I would reach for other tools first, like structured types and access or replacement tools that used types (subsindets, evalindets, indets, etc).

@tomleslie 

My edits were not intended as a criticism, and I hope weren't taken that way. Apologies it it was. I understand quite well that you were going for exposition and a prototype.

Having said that, the changes were quite short though the speedup is considerable. And its executable statements are just as understandable IMO (barring removal of comments). Things like making the mask creation done with evalhf, having the mask be float[8] datatype, some memory savings by inplace operations, and using `zip` rather than .~ for applying the mask.

It's almost fast enough for the process to be done with Explore, with parameters for the `width` and `centrefreq`. Fast experimentation of the filter application is one reason why I think fast can be useful.

(I am reminded again of a wish for fast, effective, and flexible peak detection in Maple.)

@tomleslie Nice job.

Attached is a revision that runs a bit faster.

Download ImProc_faster.mw

@ThU There is also keyboard shortcut for indexed underscripts, Ctl-Shift-_ (control-shift-underscore) for 2D Input mode.

There are also items on the Layout Palette for both indexed and literal name subscripts. (Those palette items have tooltips that show which is which).

Using either keyboard shortcut or palette entry you can obtain nice typeset subscripted input for indexed subscripts, without having to see the 1D style square-bracketed name in the input, and without having to change any preferences.

@vv Yes, that's why I called it "hmm". As in "things that make you go `hmm`". 

@Ian Jones Using Maple 2017.3 and the change-of-variables gets the following (even without conversion to MeijerG).

I find it interesting what happens to the imaginary component, without the change of variables.

restart;

ig:=cos(alpha*s)*(1/8*I)*(-HankelH1(0, beta*s)-(2*I)*BesselK(0, beta*s)/Pi)/beta^2:

ans:=value(IntegrationTools:-Change(Int(ig,s=0..infinity),s=sqrt(t),[t])):

lprint(ans);
   1/8*(I*beta^2*((-alpha^2+beta^2)/beta^2)^(1/2)*((alpha^2+beta^2)/beta^2)^(1/2)+alpha^2-beta^2)/beta^3/(alpha^2-beta^2)/((alpha^2+beta^2)/beta^2)^(1/2)

evalf(eval(ans,[alpha=1,beta=3/2]));
                     0.03081667756 - 0.04969039945 I

hmm:=value(Int(ig,s=0..infinity)):

lprint(hmm);                                                                       
   1/8/beta^3/((alpha^2+beta^2)/beta^2)^(1/2)

evalf(eval(hmm,[alpha=1,beta=3/2]));                                               
                              0.03081667755

evalf(Int(eval(ig,[alpha=1,beta=3/2]),s=0..infinity));
                       0.03081667757 - 0.04969039950 I

int( 1/8*(cos(alpha*s)*(-BesselJ(0,beta*s))/beta^2), s=0..infinity );
                                    0

evalf(Int( eval( 1/8*(cos(alpha*s)*(-BesselJ(0,beta*s))/beta^2),
                 [alpha=1,beta=3/2] ), s=0..infinity ));
                              -0.04969039950

[edited] It looks like some problem with a mix of floats and symbolics.

I originally guessed a floatPi issue, too hastily and wrongly, away from actual Maple.

It would be interesting to dig down into why `evalf/int` is using such different series, according to the presence of the float.

@Daniel Skoog Thanks Daniel.

I've just deleted a Comment of my own above, where I went created and injected such annotations manually into the PLOT structure. Of course your method of doing this is much better.

I coded my version because... I did not see any mention or examples of annotations on the Maple 2017.3 help page for the plots:-pointplot command. Could that be remedied, with both description and example?

ps. Plot annotations do not work in my Maple 2017.2 or Maple 2017.3 for 64bit ubuntu 14.04. Java complains about "TRANSLUCENT translucency" in the console. Is it confirmed to work OK with ubuntu 16.04 ?