MAGIC WP4 and WP6 hyint

Refined for v.2.0 and available for demo. Should work on 1975-2100 period and daily data so it takes too long for real time calculation.
TODO: It keeps a separate file with parameter settings (hyint_parameters.R) because of difficulties in introducing matrices into yml.

Some final changes are ongoing to simplify the code. Documentation and provenance were added.

The R dependencies have not been added to the file esmvaltool/install/R/r_requirements.txt, so when I try to run the diagnostic script it crashes with the following error message:
Error in library(ncdf4) : there is no package called ‘ncdf4’
Execution halted

Please wait  before new tests for a new version to be uploaded shortly

Tested again on Jasmin, mostly good results, but recipe option selregions: [4] and selregions: [7] crash with
in/hyint_ACCESS1-0_historical-rcp85_r1i1p1_2000_2010_ALL_tseries_.nc"
[1] "hyint: starting figures"
[1] "No value for range: assigning min and max"

 *** caught bus error ***
address 0x7f5e6b593090, cause 'non-existent physical address'

Traceback:
 1: axis(side = side, at = at, labels = labels, ...)
 2: Axis.default(...)
 3: Axis(...)
 4: localAxis(if (is.null(y)) xy$x else x, side = 1, ...)
 5: plot.default(time, type = "n", ylim = c(tmp.levels[1], tmp.levels[length(tmp.levels)]),     xlim = xlim, xlab = "Year", ylab = ylab, main = title_unit_m[ifield,         3])
 6: plot(time, type = "n", ylim = c(tmp.levels[1], tmp.levels[length(tmp.levels)]),     xlim = xlim, xlab = "Year", ylab = ylab, main = title_unit_m[ifield,         3])
 7: hyint_plot_trends(work_dir, plot_dir, ref_idx, seas)
An irrecoverable exception occurred. R is aborting now ...

and selregions: [11] crashes with
[1] "Working on  NA"
Error in regions[iselreg, ] : subscript out of bounds
Calls: hyint_trends -> calc_region_timeseries -> which
Execution halted

Ok, looking into this, thanks.
Il Lun 25 Feb 2019, 21:13 Bouwe Andela<EMAIL_ADDRESS>ha
scritto:

Tested again on Jasmin, mostly good results, but recipe option selregions:
[4] and selregions: [7] crash with
in/hyint_ACCESS1-0_historical-rcp85_r1i1p1_2000_2010_ALL_tseries_.nc"
[1] "hyint: starting figures"
[1] "No value for range: assigning min and max"
*** caught bus error ***
address 0x7f5e6b593090, cause 'non-existent physical address'
Traceback:
1: axis(side = side, at = at, labels = labels, ...)
2: Axis.default(...)
3: Axis(...)
4: localAxis(if (is.null(y)) xy$x else x, side = 1, ...)
5: plot.default(time, type = "n", ylim = c(tmp.levels[1], tmp.levels[length(tmp.levels)]),     xlim = xlim, xlab = "Year", ylab = ylab, main = title_unit_m[ifield,         3])
6: plot(time, type = "n", ylim = c(tmp.levels[1], tmp.levels[length(tmp.levels)]),     xlim = xlim, xlab = "Year", ylab = ylab, main = title_unit_m[ifield,         3])
7: hyint_plot_trends(work_dir, plot_dir, ref_idx, seas)
An irrecoverable exception occurred. R is aborting now ...
and selregions: [11] crashes with
[1] "Working on  NA"
Error in regions[iselreg, ] : subscript out of bounds
Calls: hyint_trends -> calc_region_timeseries -> which
Execution halted
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/ESMValGroup/ESMValTool/pull/754#issuecomment-467165163,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AblrsW5EHfCk6E7CPTn8A-wCY3qO-iIVks5vREPWgaJpZM4ZMeu2
.

I have added a check on user settings wrongly assigned user settings for selregions, selfields and plot_type. This resolves error for selregions: [11].
However, I have tried various settings with selregions: [4, 7] but it never crashed. Can you share the full recipe you used?
Thanks

Tested again on Jasmin, mostly good results, but recipe option selregions: [4] and selregions: [7] crash with
in/hyint_ACCESS1-0_historical-rcp85_r1i1p1_2000_2010_ALL_tseries_.nc"
[1] "hyint: starting figures"
[1] "No value for range: assigning min and max"

 *** caught bus error ***
address 0x7f5e6b593090, cause 'non-existent physical address'

Traceback:
 1: axis(side = side, at = at, labels = labels, ...)
 2: Axis.default(...)
 3: Axis(...)
 4: localAxis(if (is.null(y)) xy$x else x, side = 1, ...)
 5: plot.default(time, type = "n", ylim = c(tmp.levels[1], tmp.levels[length(tmp.levels)]),     xlim = xlim, xlab = "Year", ylab = ylab, main = title_unit_m[ifield,         3])
 6: plot(time, type = "n", ylim = c(tmp.levels[1], tmp.levels[length(tmp.levels)]),     xlim = xlim, xlab = "Year", ylab = ylab, main = title_unit_m[ifield,         3])
 7: hyint_plot_trends(work_dir, plot_dir, ref_idx, seas)
An irrecoverable exception occurred. R is aborting now ...

and selregions: [11] crashes with
[1] "Working on  NA"
Error in regions[iselreg, ] : subscript out of bounds
Calls: hyint_trends -> calc_region_timeseries -> which
Execution halted

I have added a check on wrongly assigned user settings for selregions, selfields and plot_type. This resolves error for selregions: [11].

Can you please push it to GitHub so I can test?

However, I have tried various settings with selregions: [4, 7] but it never crashed. Can you share the full recipe you used?

I just ran the recipe again with selregions: [4] (all other settings the same as the default) and it completed successfully, so this was probably some issue with the Jasmin cluster.

Version with checks on input parameters uploaded. This version includes also several minor refinements to the figures. I can also implement system2 as done for the quantilebias if we have time.

I can also implement system2 as done for the quantilebias if we have time.

It would be great if you could do it with the same function as implemented there

7b74721 is failing because input is at time a string containing further arguments in ncdf_opener_universal:
  if (interp2grid) {       print(paste("Remapping with CDO on", grid, "grid"))       if (is.null(namevar)) {         namefile <- cdo(remap_method, args = grid, input = namefile,                         output = tempfile())       } else {         namefile <- cdo(remap_method, args = grid,                         input = paste0("-selvar,", namevar, " ", namefile),                         output = tempfile())       }     }
Looking at solution.

Yes, to be expected, see my comment https://github.com/ESMValGroup/ESMValTool/pull/750#issuecomment-469326419

From our point of view this is ready to be merged. If needed, we will apply further refinements in a separate branch.

The regridding for field_exp in hyint_plot_maps.R tries to regrid to a netCDF file and not to a file with grid information:
https://github.com/ESMValGroup/ESMValTool/blob/b9466958cc1c9be0ecd4d52bd03f6b32f547a9c7/esmvaltool/diag_scripts/hyint/hyint_plot_maps.R#L136-L138
I suggest to replace the regridding of field_exp with:
# regrid when comparing
ref_filename_grid <- paste0(tools::file_path_sans_ext(ref_filename), ".grid")
cdo("griddes", input = ref_filename, stdout = ref_filename_grid)
field_exp <- ncdf_opener(filename, field, "lon", "lat", rotate = "no",
                         interp2grid = T, grid = ref_filename_grid)

After regridding the field_exp, I end up with a bunch of empty plots where field_exp is used for creating the map plot. I'm not sure yet where it is going wrong, but it appears that the field_exp variable ends up having no data in it.

The regridding for field_exp in hyint_plot_maps.R tries to regrid to a netCDF file and not to a file with grid information:
Using a reference netcdf file to specify grid information is valid for cdo (it will regrid to match the grid of the reference file).
I do not quite understand, did you test the recipe and it did not work for you? With which settings ?

Sorry didn't know that was valid cdo syntax. Our test script creates spaces in the path for some settings, which both the diagnostic script and cdo don't really like, so was working around that.
Would there be any reason why the syntax I suggested would result in empty field_exp and the original not (I'll test it on jasmin by using the original syntax with relative paths of that produces valid field_exp plots)?

I changed the ncdf_opener calls in hyint_plot_maps.R to use relative paths locally and used the netCDF file for the remapping, but I'm still running into the empty plot issue. For example:
[1] "/group_workspaces/jasmin2/cp4cds1/magic-test/tmp-ronald/REFACTORING_hyint_local/output_dir/recipe_hyint_norm_years_[2004, 2006]_20190305_110034/plots/hyint/main/dsl_norm_CCSM4_historical-rcp85_r1i1p1_2000-2010_ALL_myearmean_EU_map.png"
[1] "region:  East-Asia"
[1] "hyint: plotting data for  East-Asia-2000-2010"
[1] "No value for range: assigning min and max"

and then finally it crashes in the code where the plot levels are calculated from tmp_field_max because tmp_field_max evaluates to Inf:
Error in seq.default(-tmp_field_max, tmp_field_max, len = nlev) :
  'from' must be a finite number
Calls: hyint_plot_maps -> seq -> seq.default

If you have ideas that would help debugging that would be helpful

OK, thanks, I now understand you have issues we cannot see.
I'll try to reproduce your situations and see what happens.
On Wed, 6 Mar 2019 at 11:58, Ronald van Haren<EMAIL_ADDRESS>wrote:

I changed the ncdf_opener calls in hyint_plot_maps.R to use relative paths
locally and used the netCDF file for the remapping, but I'm still running
into the empty plot issue. For example:
[1] "/group_workspaces/jasmin2/cp4cds1/magic-test/tmp-ronald/REFACTORING_hyint_local/output_dir/recipe_hyint_norm_years_[2004, 2006]_20190305_110034/plots/hyint/main/dsl_norm_CCSM4_historical-rcp85_r1i1p1_2000-2010_ALL_myearmean_EU_map.png"
[1] "region:  East-Asia"
[1] "hyint: plotting data for  East-Asia-2000-2010"
[1] "No value for range: assigning min and max"
and then finally it crashes in the code where the plot levels are
calculated from tmp_field_max because tmp_field_max evaluates to Inf:
Error in seq.default(-tmp_field_max, tmp_field_max, len = nlev) :
'from' must be a finite number
Calls: hyint_plot_maps -> seq -> seq.default
If you have ideas that would help debugging that would be helpful
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/ESMValGroup/ESMValTool/pull/754#issuecomment-470064303,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AblrsUEzP0aSb9s7SjFp-W9LQ8ATQn6Eks5vT59pgaJpZM4ZMeu2
.

Hi @rvanharen , in order for us to reproduce the issue, could you provide us the recipe created by your test script ? (actually spaces in a unix path do sound a bit problematic indeed)

I changed the ncdf_opener calls in hyint_plot_maps.R to use relative paths locally and used the netCDF file for the remapping, but I'm still running into the empty plot issue. For example:
[1] "/group_workspaces/jasmin2/cp4cds1/magic-test/tmp-ronald/REFACTORING_hyint_local/output_dir/recipe_hyint_norm_years_[2004, 2006]_20190305_110034/plots/hyint/main/dsl_norm_CCSM4_historical-rcp85_r1i1p1_2000-2010_ALL_myearmean_EU_map.png"
[1] "region:  East-Asia"
[1] "hyint: plotting data for  East-Asia-2000-2010"
[1] "No value for range: assigning min and max"

and then finally it crashes in the code where the plot levels are calculated from tmp_field_max because tmp_field_max evaluates to Inf:
Error in seq.default(-tmp_field_max, tmp_field_max, len = nlev) :
  'from' must be a finite number
Calls: hyint_plot_maps -> seq -> seq.default

If you have ideas that would help debugging that would be helpful

Since
tmp_field_max <- max(abs(tmp_field), na.rm = T)
you must have Inf in the tmp_field, withtmp_field <- field_exp - field_ref. Can you check the field_exp and field_ref soon after reading them? You can use is.infinite(x). I see no issue in my data.

I changed the ncdf_opener calls in hyint_plot_maps.R to use relative paths locally and used the netCDF file for the remapping, but I'm still running into the empty plot issue. For example:
[1] "/group_workspaces/jasmin2/cp4cds1/magic-test/tmp-ronald/REFACTORING_hyint_local/output_dir/recipe_hyint_norm_years_[2004, 2006]_20190305_110034/plots/hyint/main/dsl_norm_CCSM4_historical-rcp85_r1i1p1_2000-2010_ALL_myearmean_EU_map.png"
[1] "region:  East-Asia"
[1] "hyint: plotting data for  East-Asia-2000-2010"
[1] "No value for range: assigning min and max"

and then finally it crashes in the code where the plot levels are calculated from tmp_field_max because tmp_field_max evaluates to Inf:
Error in seq.default(-tmp_field_max, tmp_field_max, len = nlev) :
  'from' must be a finite number
Calls: hyint_plot_maps -> seq -> seq.default

If you have ideas that would help debugging that would be helpful

Since
tmp_field_max <- max(abs(tmp_field), na.rm = T)
you must have Inf in the tmp_field, withtmp_field <- field_exp - field_ref. Can you check the field_exp and field_ref soon after reading them? You can use is.infinite(x). I see no issue in my data.

Running with default setting (as I can partially guess from the filename
you quote) works fine. You can see our output here:
http://wilma.to.isac.cnr.it/arnone/temp/esmvaltool_output/recipe_hyint_20190306_112916/plots/hyint/main/
On Wed, 6 Mar 2019 at 12:09, Jost von Hardenberg<EMAIL_ADDRESS>wrote:

Hi @rvanharen https://github.com/rvanharen , in order for us to
reproduce the issue, could you provide us the recipe created by your test
script ? (actually spaces in a unix path do sound a bit problematic indeed)
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/ESMValGroup/ESMValTool/pull/754#issuecomment-470067729,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AblrsaKtZcGW1Vuj6NLaoQwWN4LCh_o1ks5vT6H3gaJpZM4ZMeu2
.

Hi @rvanharen , in order for us to reproduce the issue, could you provide us the recipe created by your test script ? (actually spaces in a unix path do sound a bit problematic indeed)

add_trend: true
add_trend_sd: true
add_trend_sd_shade: true
add_tseries_lines: true
add_zeroline: true
autolevels: true
autolevels_scale: 1
autolevels_scale_t: 1
boxregion: false
input_files: ['/group_workspaces/jasmin2/cp4cds1/magic-test/tmp-ronald/REFACTORING_hyint_local/output_dir/recipe_hyint_norm_years_[2004,
    2006]_20190305_110034/preproc/hyint/pr/metadata.yml']
lm_trend: true
log_level: info
max_data_filesize: 100
norm_years: [2004, 2006]
npancol: 2
npanrow: 3
oplot_grid: false
output_file_type: png
plot_dir: /group_workspaces/jasmin2/cp4cds1/magic-test/tmp-ronald/REFACTORING_hyint_local/output_dir/recipe_hyint_norm_years_[2004,
  2006]_20190305_110034/plots/hyint/main
plot_type: [1, 2, 3, 11, 12, 13, 14, 15]
profile_diagnostic: false
recipe: recipe_hyint_norm_years_[2004, 2006].yml
removedesert: false
rgrid: false
run_dir: /group_workspaces/jasmin2/cp4cds1/magic-test/tmp-ronald/REFACTORING_hyint_local/output_dir/recipe_hyint_norm_years_[2004,
  2006]_20190305_110034/run/hyint/main
scale100years: true
scalepercent: false
script: main
selfields: [8, 4, 1, 9, 3, 2]
selregions: [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
trend_years: false
trend_years_only: false
version: 2.0a1
weight_tseries: true
work_dir: /group_workspaces/jasmin2/cp4cds1/magic-test/tmp-ronald/REFACTORING_hyint_local/output_dir/recipe_hyint_norm_years_[2004,
  2006]_20190305_110034/work/hyint/main
write_netcdf: true
write_plots: true

I see no issue in my data.

Could you please check with the CP4CDS dataset on Jasmin? Because that is the data that the diagnostic should work with for the MAGIC project.

@rvanharen what you list is the settings.yml file, does your test script produce directly those or rather a recipe*yml file ? what is in your metadata.yml file ? I would like to be able to reproduce you error before giving an opinion.

@jhardenberg I did update the paste to the recipy shortly after I pasted the settings.yml

you must have Inf in the tmp_field, withtmp_field <- field_exp - field_ref. Can you check the field_exp and field_ref soon after reading them? You can use is.infinite(x). I see no issue in my data.

After reading there is quite a bit of data in both field_exp and field_ref (btw I have no issues with field_ref, that one seems to contain data). I seem to 'lose' the data when the multiyear average is extracted:
          # MAPS: select required year (if requested, multiyear average
          #       is stored at iyear=1)
          print(paste0('field_exp max before multiyear average:', max(is.finite(field_exp), na.rm=T)))
          print(paste0('field_ref max before multiyear average:', max(is.finite(field_ref), na.rm=T)))

          field_ref <- field_ref[, , iyear]
          field_exp <- field_exp[, , iyear_ref]
          tmp_field <- field_exp
          print(paste0('field_exp max after multiyear average:', max(is.finite(field_exp), na.rm=T)))
          print(paste0('field_ref max after multiyear average:', max(is.finite(field_ref), na.rm=T)))

produces the following output:
[1] "field_exp max before multiyear average:1"
[1] "field_ref max before multiyear average:1"
[1] "field_exp max after multiyear average:0"
[1] "field_ref max after multiyear average:1"
[1] "No value for range: assigning min and max"

Hi, I just gave a look. The problem, as you mentioned comes from the fact that your script generates directory names like "recipe_hyint_plot_type_[2, 3, 12, 13, 14, 15]20190306_12075"
Rather than changing the diagnostic would it not be easier to fix the test script so that it avoids spaces in the directory names (this is a terrible idea anyway in my opinion ...) ? You found this problem in this diagnostic but I cannot guarantee that also other may not have objections to such funky directory names. The problem never came out before, because by default esmvaltool does not generate directory names with spaces (unless you call your diagnostic "hyint_plot_type[2, 3, 12, 13, 14, 15]" of course ....)

Spaces in the fullpath is IMO a valid use case. I already changed the plot_maps script to use relative paths, but I can do the same with the rest of the scripts and see if that solves the problem.
In addition I will rerun the test using a path without spaces and report back if that solves it, but I imagine that something would have complained already as I have it locally single quoted or as relative path.

I changed the recipy name to localtest.yml (so now there are no spaces in the path) and ran it again, but the issue is still the same on jasmin. This is the command I used to run it:
esmvaltool -c /group_workspaces/jasmin2/cp4cds1/magic-test/tmp-ronald/config_user.yml "/group_workspaces/jasmin2/cp4cds1/magic-test/tmp-ronald/REFACTORING_hyint_local/localtest.yml" --skip-nonexistent

OK, I reproduced the error. It is related to the selection of normalization years in an interval falling within the range of available years and not at the beginning as it is usually adopted. Will fix asap.

Ok guys, I implemented the same changes as in #750 for the cdo function. It now puts quotes around all filenames and so it now accepts also dirnames and filenames with spaces. I also followed @bouweandela 's suggestion and split all chained cdo commands into separate commands. Temp files are now generated automatically if no "output" is specified.
It turns out that while most commands work, a few cdo commands (e.g. delete and copy) do not like input files with space (with quotes or without). Since we had only one of those I used a clumsy but effective fix (adding 0 to the file and copying it to a temporary file first) in that only case ....
@rvanharen could you please try again your test script with a fresh pull of this branch (without any of the later modifications which you were testing). It should simply work now.

Quick comment on this: as already noted, cdo with separate commands may
take quite some time more to perform the requested operations so we should
then find a better way to deal with this.
On Thu, 7 Mar 2019 at 00:42, Jost von Hardenberg<EMAIL_ADDRESS>wrote:

Ok guys, I implemented the same changes as in #750
https://github.com/ESMValGroup/ESMValTool/pull/750 for the cdo
function. It now puts quotes around all filenames and so it now accepts
also dirnames and filenames with spaces. I also followed @bouweandela
https://github.com/bouweandela 's suggestion and split all chained cdo
commands into separate commands. Temp files are now generated automatically
if no "output" is specified.
It turns out that while most commands work, a few cdo commands (e.g.
delete and copy) do not like input files with space (with quotes or
without). Since we had only one of those I used a clumsy but effective fix
(adding 0 to the file and copying it to a temporary file first) in that
only case ....
@rvanharen https://github.com/rvanharen could you please try again your
test script with a fresh pull of this branch (without any of the later
modifications which you were testing). It should simply work now.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/ESMValGroup/ESMValTool/pull/754#issuecomment-470322459,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AblrsbmFdQPSNBzU5LsEPzfb_Vv9fCPtks5vUFJ_gaJpZM4ZMeu2
.

Hi @rvanharen where/when did you find this error? I tried the latest version of the hyint recipe using a diagnostic name in the recipe with spaces (which creates directory names with spaces) and it seems to work nicely.
Also I tried, and for me
cdo remapcon,'my grid file.nc' input.nc out.nc
works... (I tried both with cdo 1.9.3 and 1.9.6)

The problem was apparently a comma in the filename..

Ah, ok, thank you, I managed indeed to reproduce the error choosing a name with spaces and commas for the diagnostic in the recipe.
@rvanharen thank you for the suggestion with relative paths. Since the issue actually only appears when remapcon2 is used, I have chosen to implement an even simpler fix by doing a previous "cdo griddes". This now seems to work for me.

I realized that we were not testing all options in the documentation, so I've run some additional tests. There were two cases in which chaning an option from the default resulted in a crash:

setting lm_trend to false:

[1] "******** PLOT TYPE: 14 *********"
[1] 960 800
[1] "HyInt_trends: reading file  /group_workspaces/jasmin2/cp4cds1/magic-test/tmp-bandela/REFACTORING_hyint/output_dir/recipe_hyint_lm_trend_False_20190323_055259/work/hyint/main/hyint_ACCESS1-0_historical-rcp85_r1i1p1_2000_2010_ALL_tseries_.nc"
[1] "hyint: starting figures"
[1] "No value for range: assigning min and max"
[1] "No value for range: assigning min and max"
Error in plot.window(...) : need finite 'ylim' values
Calls: hyint_plot_trends -> plot -> plot.default -> localWindow -> plot.window
In addition: There were 19 warnings (use warnings() to see them)
Execution halted

setting removedesert to true

[1] "Reading experiment  /group_workspaces/jasmin2/cp4cds1/magic-test/tmp-bandela/REFACTORING_hyint/output_dir/recipe_hyint_removedesert_True_20190323_054118/work/hyint/main/hyint_MPI-ESM-MR_historical-rcp85_r1i1p1_2000_2010_ALL.nc"
Error in field_exp * retdes3D : non-conformable arrays
Calls: hyint_plot_maps
In addition: Warning messages:
1: In dir.create(plot_dir_exp, recursive = T) :
  '/group_workspaces/jasmin2/cp4cds1/magic-test/tmp-bandela/REFACTORING_hyint/output_dir/recipe_hyint_removedesert_True_20190323_054118/plots/hyint/main' already exists
2: In dir.create(plot_dir_exp, recursive = T) :
  '/group_workspaces/jasmin2/cp4cds1/magic-test/tmp-bandela/REFACTORING_hyint/output_dir/recipe_hyint_removedesert_True_20190323_054118/plots/hyint/main' already exists
3: In dir.create(plot_dir_exp, recursive = T) :
  '/group_workspaces/jasmin2/cp4cds1/magic-test/tmp-bandela/REFACTORING_hyint/output_dir/recipe_hyint_removedesert_True_20190323_054118/plots/hyint/main' already exists
Execution halted

Restored removedesert code lost during cleanup. Removed option lm_trend. Fixed a bug for calendar 360_day models. Testing again on all cmip5 models.

Thank you for making the recipe options more user friendly. I have tested again and noticed that the issue with removedesert: true is still present:
[1] "cdo  remapcon2,/tmp/RtmpTMxPUl/file2d32563515239 '/tmp/RtmpTMxPUl/file2d3252b502a8b' /tmp/RtmpTMxPUl/file2d32521fb80bc "
cdo remapcon2: SCRIP second order conservative weights from gaussian (192x96) to lonlat (192x145) grid, with source mask (3516)
cdo remapcon2: Processed 755712 values from 1 variable over 41 timesteps [1.63s 3224MB]
 num [1:192, 1:145, 1:41] NA NA NA NA NA NA NA NA NA NA ...
NULL
 num [1:192, 1:96, 1:41] NA NA NA NA NA NA NA NA NA NA ...
NULL
Error in field_exp * exp_retdes3D : non-conformable arrays
Calls: hyint_plot_maps
In addition: Warning messages:
1: In dir.create(plot_dir_exp, recursive = T) :
  '/group_workspaces/jasmin2/cp4cds1/magic-test/tmp-bandela/REFACTORING_hyint/output_dir/recipe_hyint_removedesert_True_20190329_170837/plots/hyint/main' already exists
2: In dir.create(plot_dir_exp, recursive = T) :
  '/group_workspaces/jasmin2/cp4cds1/magic-test/tmp-bandela/REFACTORING_hyint/output_dir/recipe_hyint_removedesert_True_20190329_170837/plots/hyint/main' already exists
Execution halted

I also noticed that the memory required for running the recipe has increased a lot since the last time I tested.

Thanks for spotting this. Fixed removedesert for plot_type=2 and rgrid=F option. More memory I guess because I increased the pre-set time interval to 1980-2020 to have a more meaningful example. If too large we can reduce it again.

Thanks I will test again. The larger memory is no problem if it is as expected.

It is still not working. Most tests fail with a message like this:
[1] "******** PLOT TYPE: 2 *********"
[1] "Reading reference  /group_workspaces/jasmin2/cp4cds1/magic-test/tmp-bandela/REFACTORING_hyint/output_dir/recipe_hyint_autolevels_scale_t_1.5_20190418_115310/work/hyint/main/hyint_ACCESS1-0_historical-rcp85_r1i1p1_1980_2020_ALL.nc"
[1] "Reading experiment  /group_workspaces/jasmin2/cp4cds1/magic-test/tmp-bandela/REFACTORING_hyint/output_dir/recipe_hyint_autolevels_scale_t_1.5_20190418_115310/work/hyint/main/hyint_MPI-ESM-MR_historical-rcp85_r1i1p1_1980_2020_ALL.nc"
[1] "Reading experiment  /group_workspaces/jasmin2/cp4cds1/magic-test/tmp-bandela/REFACTORING_hyint/output_dir/recipe_hyint_autolevels_scale_t_1.5_20190418_115310/work/hyint/main/hyint_MPI-ESM-MR_historical-rcp85_r1i1p1_1980_2020_ALL.nc"
[1] "Reading experiment  /group_workspaces/jasmin2/cp4cds1/magic-test/tmp-bandela/REFACTORING_hyint/output_dir/recipe_hyint_autolevels_scale_t_1.5_20190418_115310/work/hyint/main/hyint_MPI-ESM-MR_historical-rcp85_r1i1p1_1980_2020_ALL.nc"
[1] "Reading experiment  /group_workspaces/jasmin2/cp4cds1/magic-test/tmp-bandela/REFACTORING_hyint/output_dir/recipe_hyint_autolevels_scale_t_1.5_20190418_115310/work/hyint/main/hyint_MPI-ESM-MR_historical-rcp85_r1i1p1_1980_2020_ALL.nc"
[1] "Reading experiment  /group_workspaces/jasmin2/cp4cds1/magic-test/tmp-bandela/REFACTORING_hyint/output_dir/recipe_hyint_autolevels_scale_t_1.5_20190418_115310/work/hyint/main/hyint_MPI-ESM-MR_historical-rcp85_r1i1p1_1980_2020_ALL.nc"
[1] "Reading experiment  /group_workspaces/jasmin2/cp4cds1/magic-test/tmp-bandela/REFACTORING_hyint/output_dir/recipe_hyint_autolevels_scale_t_1.5_20190418_115310/work/hyint/main/hyint_MPI-ESM-MR_historical-rcp85_r1i1p1_1980_2020_ALL.nc"
[1] "hyint: starting figures"
[1]  672 1260
[1] "region:  World"
[1] "hyint: plotting data for  World-1980-2020"
[1] "No value for range: assigning min and max"
Error in .filled.contour(as.double(x), as.double(y), z, as.double(levels),  :
  dimension mismatch
Calls: hyint_plot_maps -> filled_contour3 -> .filled.contour
In addition: Warning messages:
1: In dir.create(plot_dir_exp, recursive = T) :
  '/group_workspaces/jasmin2/cp4cds1/magic-test/tmp-bandela/REFACTORING_hyint/output_dir/recipe_hyint_autolevels_scale_t_1.5_20190418_115310/plots/hyint/main' already exists
2: In dir.create(plot_dir_exp, recursive = T) :
  '/group_workspaces/jasmin2/cp4cds1/magic-test/tmp-bandela/REFACTORING_hyint/output_dir/recipe_hyint_autolevels_scale_t_1.5_20190418_115310/plots/hyint/main' already exists
Execution halted

Could you please test it a bit more thorough yourself and let me know when it works? You can find a list of the options that I try here: https://github.com/ESMValGroup/ESMValTool/blob/version2_recipe_settings_test_utility/esmvaltool/utils/testing/recipe_settings/options.yml#L242

Restored missing code lines for regridding, added further checks on user options. Tested on all options, several models and combinations. We are working on adding provenance to figures and inclusion of data from the extremeEvents recipe in a separate branch.

I tested with the latest updates, things have improved, but the issue with cdo remapcon2 is back, see https://github.com/ESMValGroup/ESMValTool/pull/754#pullrequestreview-212809820 and https://github.com/ESMValGroup/ESMValTool/pull/754#issuecomment-471757393

@bouweandela sorry, will update soon for remapcon2. We also have the provenance added to figures, I would push that too.

Sure, that's fine.

With the latest changes, setting the recipe option rgrid: r100x50 fails:
[1] "hyint: starting routine"
[1] "hyint: pre-processing file: /var/scratch2/bandela/test/version2_development/output_dir/recipe_hyint_rgrid_r100x50_20190508_094403/preproc/hyint/pr/CMIP5_ACCESS1-0_day_historical-rcp85_r1i1p1_pr_1980-2020.nc"
[1] "cdo  griddes 'r100x50'   > '/tmp/RtmpGiJoHz/file73a171d76bf1'"
cdo griddes: Open failed on >r100x50<
No such file or directory
Error in cdo("griddes", input = rgrid, stdout = gridf) :
  Failed ( 1 ): cdo  griddes 'r100x50'   > '/tmp/RtmpGiJoHz/file73a171d76bf1'
Calls: hyint_preproc -> cdo
Execution halted

We are discussing whether to drop this artificial split of the recipe of WP4 and WP6, Andreas is ok. @nielsdrost, shall we drop it and keep only one tool for the portal too? Then the second separate recipe can be dropped and the different parameters in the recipes can be put together again.

Hi Enrico,
Niels is on holiday, but I think it would be fine to make a single recipe if that is scientifically most meaningful. However, I would strongly prefer to merge this pull request before adding any new features. I'm almost done testing for the tenth time or so and it looks like recipe_hyint.yml is finally working now, so please don't push additional changes.

Or is there already a substantial amount of code in this related to recipe_hyint_trends.yml? Because then I will stop testing now and you can let me know when everything works and I will test again.

Ok, I tested recipe_hyint.yml and everything seems to work now, but see my questions above.

OK, great. Yes, we should go ahead with keeping only one recipe, the split was artificial only for the workpackages in the project. This is actually just exactly as it was before, the second recipe I added was a user-friendly version calling only the timeseries and trends plots (i.e. plot_type = 11:15) with all regions selected. So there is no change at all to the code, nor to your testing (you already use all possibilities). I would just push a recipe_hyint.yml with default settings that include all plot_types values and remove the recipe_hyint_trends.yml. I will have to push then at least 10 drinks for you.


Display item profit in GE plugin

The Grand Exchange plugin already shows the price and the high alch of items in the search field. This change expands that by calculating the profit that would be obtained if the item and the nature rune were bought from the GE for the purpose of alching them. Images below show what the updates look like before and after the change.

@psikoi can you review this?

I really don't like how that interface is getting filled up like that.
I asked about it on discord and abex had a good idea about it.

I like that, too.

I reverted changes and added the parenthesis you guys suggested. Below is an image of the new implementation. I saw Adam suggest that this could be a config. I couldn't see an easy way to extend the config manager to where I need to reference it. I'll keep at it if you guys think it's important.

It could be good if the profit/loss was correctly colored (green/red)

Are you thinking of just the profit/loss portion being colored or the whole string, alch amount and all?

Just the difference

I added a new label so I could color code just the profit margin. Here is the result:

The code looks fine, apart from what I commented. I haven't tested yet.
I'm also not very happy with the amount of text that component is displaying. But unless someone has a solution, I guess it makes sense to have the profit margin.


⚠ Mobitopup has degraded performance

In 7ad5c1f, Mobitopup (https://napcard.voz.tech) experienced degraded performance:

HTTP code: 200
Response time: 8274 ms

Resolved: Mobitopup performance has improved in dd4a3e6.


what does the letter P stand for?

in other bt client p stand for peer, only include the unfinished users.
elementum's letter P include both the fished seeds and the unfinished peers according to the tracker. Is it possible to subtract the number of seeds from P?

@i990049

Is it possible to subtract the number of seeds from P?

for 1 (1!!!!!!) file torrent it already does do that.
peersTotal := max(ts.GetListPeers()-ts.GetListSeeds()
see https://github.com/elgatito/elementum/blob/638d5c307b53763e752a698252ecb03de06e085a/bittorrent/torrent.go#L323-L327
	seeds := ts.GetNumSeeds()
	seedsTotal := max(ts.GetListSeeds(), ts.GetNumComplete())

	peers := ts.GetNumPeers() - ts.GetNumSeeds()
	peersTotal := max(ts.GetListPeers()-ts.GetListSeeds(), ts.GetNumComplete()+ts.GetNumIncomplete())

then open https://www.libtorrent.org/reference-Torrent_Status.html#torrent_status
num_seeds - the number of peers that are seeding that this client is currently connected to.
num_peers - the number of peers this torrent currently is connected to.
list_seeds - the number of seeds in our peer list
list_peers - the total number of peers (including seeds)
num_complete - the total number of peers that have the whole file
num_incomplete - the total number of peers that are still downloading
BUT since elementum can download 1 file from torrent that has many files, it considers peer that this file as seed, even if this peer has only this file and thus considered by tracker as peer.
example: season pack, or movie with sample.
so if i have movie file, but i do not have sample file, then for tracker i am a peer, but for you (who downloads only movie file) - i am a seed.

Goldfinger 2023 BluRay 1080p x265 10bit 2Audio DDP5.1 MNHD-FRDS/
金手指.The.Goldfinger.2023.BluRay.1080p.x265.10bit.2Audio.DDP5.1.MNHD-FRDS.mkv
the torrent is one folder contain one mkv
in qbittorrent, peers=unfinished users=95-48=47
in elementum, peers=unfinished and finished users=48+47=95
is it possible to let elemenum to display unfinished users 47？

is it possible to let elemenum to display unfinished users 47？

It is possible of course. I showed you the code.
But what about multi file torrents? Read above my explanation about it. Your change will break that logic.
You will need to create complex logic for different use cases. I'm not sure that anyone want to do this.

use
num_complete
num_incomplete
will be the same as the other BT client,  no matter how many files you download.

That's up to @elgatito to decide, he introduced that logic.


Fix UTC issues

This makes all the unit tests pass even if the machine timezone is not UTC

Nice.  I noticed those tests were failing too but didn't look in to it.  I guess folks in the UTC timezone are at a bit of a disadvantage seeing errors like these.  Interesting.

Yes.
One of the issues was a typo (Month instead of Minute); I don't know how that ever passed.  (Unless the test was only run at the end of each hour?)


All the slicing stuff and evolvers

This branch makes handling configuration slices uniform in saving/retrieving measurement data and executing measurements. It also changes the proposer API into 'evolvers' which do the accept/reject themselves.
This breaks compatibility with any existing HDF5 files, HMC scripts, and measurement scripts!

We went over the proposer --> evolver change in detail.
We did a physics comparison on beta=3, U=2, nt=32, lattice=triangle, discretization=exponential and found good agreement between the new version and old version.  This required reweighting, which was a test of the slicing and dicing functionality (although we didn't go over it in detail).


Problem: Installation of Python Modules via requirements.txt doesnt seem to work as intended

Issue
Hi,
we encouter an Issue regarding installing a Python Module into Semaphore.
We want to use ARA as callback plugin for all our runs, and while this used to work, in recent changes it seem to broke.
Before it was enough to install the Module into the Ansible Repo's venv, but that no longer seems to work.
We added ara into the requirements.txt to be installed on Container start, and according to logs it is getting installed.
However on Runs triggered by the WebUI, we always get this line from Debug:
[WARNING]: Skipping plugin (/home/semaphore/.local/lib/python3.11/site-packages/ara/plugins/callback/ara_default.py), cannot load: No module named 'ara'
Which is odd - Going into the container as semaphore user, I can import the ara module just fine.
The only workarround that works so far was also installing the ara module as root, after that its working via Web perfectly fine.
Doing everything by hand within the Container or on the Host within the venv it works from CLI just fine.
This only seems to affect the web triggered Runs.
Impact
Ansible (task execution)
Installation method
Docker
Database
Postgres
Browser
Microsoft Edge
Semaphore Version
v2.9.112-4f95ac8-1717065276
Ansible Version
external venv:

ansible [core 2.16.7]
  config file = /projects/ansible/ansible.cfg
  configured module search path = ['/projects/ansible/library']
  ansible python module location = /projects/ansible/env/lib64/python3.11/site-packages/ansible
  ansible collection location = /projects/ansible/collections
  executable location = /projects/ansible/env/bin/ansible
  python version = 3.11.7 (main, Jan 22 2024, 00:00:00) [GCC 11.4.1 20231218 (Red Hat 11.4.1-3)] (/projects/ansible/env/bin/python3.11)
  jinja version = 3.1.4
  libyaml = True

semaphore web:

ansible-playbook [core 2.16.7]
  config file = /tmp/semaphore/ansible.cfg
  configured module search path = ['/tmp/semaphore/library']
  ansible python module location = /opt/semaphore/venv/lib/python3.11/site-packages/ansible
  ansible collection location = /tmp/semaphore/collections
  executable location = /opt/semaphore/venv/bin/ansible-playbook
  python version = 3.11.9 (main, Apr 14 2024, 13:40:00) [GCC 13.2.1 20231014] (/opt/semaphore/venv/bin/python3)
  jinja version = 3.1.4
  libyaml = True

Logs & errors
Podman Logs:
No additional system dependencies to install
Installing additional python dependencies
Requirement already satisfied: requests in /opt/semaphore/venv/lib/python3.11/site-packages (from -r /etc/semaphore/requirements.txt (line 1)) (2.32.3)
Collecting molecule (from -r /etc/semaphore/requirements.txt (line 2))
  Downloading molecule-24.2.1-py3-none-any.whl.metadata (5.8 kB)
Collecting redis (from -r /etc/semaphore/requirements.txt (line 4))
  Downloading redis-5.0.4-py3-none-any.whl.metadata (9.3 kB)
Collecting hvac==1.2.0 (from -r /etc/semaphore/requirements.txt (line 5))
  Downloading hvac-1.2.0-py3-none-any.whl.metadata (3.4 kB)
Collecting ara (from -r /etc/semaphore/requirements.txt (line 6))
  Downloading ara-1.7.1-py2.py3-none-any.whl.metadata (9.0 kB)
Collecting pyhcl<0.5.0,>=0.4.4 (from hvac==1.2.0->-r /etc/semaphore/requirements.txt (line 5))
  Downloading pyhcl-0.4.5-py3-none-any.whl.metadata (3.9 kB)
Requirement already satisfied: charset-normalizer<4,>=2 in /opt/semaphore/venv/lib/python3.11/site-packages (from requests->-r /etc/semaphore/requirements.txt (line 1)) (3.3.2)
Requirement already satisfied: idna<4,>=2.5 in /opt/semaphore/venv/lib/python3.11/site-packages (from requests->-r /etc/semaphore/requirements.txt (line 1)) (3.7)
Requirement already satisfied: urllib3<3,>=1.21.1 in /opt/semaphore/venv/lib/python3.11/site-packages (from requests->-r /etc/semaphore/requirements.txt (line 1)) (2.2.1)
Requirement already satisfied: certifi>=2017.4.17 in /opt/semaphore/venv/lib/python3.11/site-packages (from requests->-r /etc/semaphore/requirements.txt (line 1)) (2024.2.2)
Collecting ansible-compat>=4.1.8 (from molecule->-r /etc/semaphore/requirements.txt (line 2))
  Downloading ansible_compat-24.6.0-py3-none-any.whl.metadata (2.9 kB)
Requirement already satisfied: ansible-core>=2.12.10 in /opt/semaphore/venv/lib/python3.11/site-packages (from molecule->-r /etc/semaphore/requirements.txt (line 2)) (2.16.7)
Collecting click<9,>=8.0 (from molecule->-r /etc/semaphore/requirements.txt (line 2))
  Downloading click-8.1.7-py3-none-any.whl.metadata (3.0 kB)
Collecting click-help-colors>=0.9 (from molecule->-r /etc/semaphore/requirements.txt (line 2))
  Downloading click_help_colors-0.9.4-py3-none-any.whl.metadata (4.1 kB)
Collecting enrich>=1.2.7 (from molecule->-r /etc/semaphore/requirements.txt (line 2))
  Downloading enrich-1.2.7-py3-none-any.whl.metadata (3.9 kB)
Collecting jsonschema>=4.9.1 (from molecule->-r /etc/semaphore/requirements.txt (line 2))
  Downloading jsonschema-4.22.0-py3-none-any.whl.metadata (8.2 kB)
Requirement already satisfied: Jinja2>=2.11.3 in /opt/semaphore/venv/lib/python3.11/site-packages (from molecule->-r /etc/semaphore/requirements.txt (line 2)) (3.1.4)
Requirement already satisfied: packaging in /usr/lib/python3.11/site-packages (from molecule->-r /etc/semaphore/requirements.txt (line 2)) (23.2)
Collecting pluggy<2.0,>=0.7.1 (from molecule->-r /etc/semaphore/requirements.txt (line 2))
  Downloading pluggy-1.5.0-py3-none-any.whl.metadata (4.8 kB)
Requirement already satisfied: PyYAML>=5.1 in /opt/semaphore/venv/lib/python3.11/site-packages (from molecule->-r /etc/semaphore/requirements.txt (line 2)) (6.0.1)
Collecting rich>=9.5.1 (from molecule->-r /etc/semaphore/requirements.txt (line 2))
  Downloading rich-13.7.1-py3-none-any.whl.metadata (18 kB)
Collecting wcmatch>=8.1.2 (from molecule->-r /etc/semaphore/requirements.txt (line 2))
  Downloading wcmatch-8.5.2-py3-none-any.whl.metadata (4.8 kB)
Collecting pbr!=2.1.0,>=2.0.0 (from ara->-r /etc/semaphore/requirements.txt (line 6))
  Downloading pbr-6.0.0-py2.py3-none-any.whl.metadata (1.3 kB)
Collecting cliff (from ara->-r /etc/semaphore/requirements.txt (line 6))
  Downloading cliff-4.7.0-py3-none-any.whl.metadata (2.1 kB)
Collecting subprocess-tee>=0.4.1 (from ansible-compat>=4.1.8->molecule->-r /etc/semaphore/requirements.txt (line 2))
  Downloading subprocess_tee-0.4.1-py3-none-any.whl.metadata (2.8 kB)
Requirement already satisfied: cryptography in /opt/semaphore/venv/lib/python3.11/site-packages (from ansible-core>=2.12.10->molecule->-r /etc/semaphore/requirements.txt (line 2)) (42.0.7)
Requirement already satisfied: resolvelib<1.1.0,>=0.5.3 in /opt/semaphore/venv/lib/python3.11/site-packages (from ansible-core>=2.12.10->molecule->-r /etc/semaphore/requirements.txt (line 2)) (1.0.1)
Requirement already satisfied: MarkupSafe>=2.0 in /opt/semaphore/venv/lib/python3.11/site-packages (from Jinja2>=2.11.3->molecule->-r /etc/semaphore/requirements.txt (line 2)) (2.1.5)
Collecting attrs>=22.2.0 (from jsonschema>=4.9.1->molecule->-r /etc/semaphore/requirements.txt (line 2))
  Downloading attrs-23.2.0-py3-none-any.whl.metadata (9.5 kB)
Collecting jsonschema-specifications>=2023.03.6 (from jsonschema>=4.9.1->molecule->-r /etc/semaphore/requirements.txt (line 2))
  Downloading jsonschema_specifications-2023.12.1-py3-none-any.whl.metadata (3.0 kB)
Collecting referencing>=0.28.4 (from jsonschema>=4.9.1->molecule->-r /etc/semaphore/requirements.txt (line 2))
  Downloading referencing-0.35.1-py3-none-any.whl.metadata (2.8 kB)
Collecting rpds-py>=0.7.1 (from jsonschema>=4.9.1->molecule->-r /etc/semaphore/requirements.txt (line 2))
  Downloading rpds_py-0.18.1-cp311-cp311-musllinux_1_2_x86_64.whl.metadata (4.1 kB)
Collecting markdown-it-py>=2.2.0 (from rich>=9.5.1->molecule->-r /etc/semaphore/requirements.txt (line 2))
  Downloading markdown_it_py-3.0.0-py3-none-any.whl.metadata (6.9 kB)
Collecting pygments<3.0.0,>=2.13.0 (from rich>=9.5.1->molecule->-r /etc/semaphore/requirements.txt (line 2))
  Downloading pygments-2.18.0-py3-none-any.whl.metadata (2.5 kB)
Collecting bracex>=2.1.1 (from wcmatch>=8.1.2->molecule->-r /etc/semaphore/requirements.txt (line 2))
  Downloading bracex-2.4-py3-none-any.whl.metadata (3.6 kB)
Collecting PrettyTable>=0.7.2 (from cliff->ara->-r /etc/semaphore/requirements.txt (line 6))
  Downloading prettytable-3.10.0-py3-none-any.whl.metadata (30 kB)
Collecting autopage>=0.4.0 (from cliff->ara->-r /etc/semaphore/requirements.txt (line 6))
  Downloading autopage-0.5.2-py3-none-any.whl.metadata (7.9 kB)
Collecting cmd2>=1.0.0 (from cliff->ara->-r /etc/semaphore/requirements.txt (line 6))
  Downloading cmd2-2.4.3-py3-none-any.whl.metadata (12 kB)
Collecting stevedore>=2.0.1 (from cliff->ara->-r /etc/semaphore/requirements.txt (line 6))
  Downloading stevedore-5.2.0-py3-none-any.whl.metadata (2.3 kB)
Collecting pyperclip>=1.6 (from cmd2>=1.0.0->cliff->ara->-r /etc/semaphore/requirements.txt (line 6))
  Downloading pyperclip-1.8.2.tar.gz (20 kB)
  Installing build dependencies: started
  Installing build dependencies: finished with status 'done'
  Getting requirements to build wheel: started
  Getting requirements to build wheel: finished with status 'done'
  Installing backend dependencies: started
  Installing backend dependencies: finished with status 'done'
  Preparing metadata (pyproject.toml): started
  Preparing metadata (pyproject.toml): finished with status 'done'
Collecting wcwidth>=0.1.7 (from cmd2>=1.0.0->cliff->ara->-r /etc/semaphore/requirements.txt (line 6))
  Downloading wcwidth-0.2.13-py2.py3-none-any.whl.metadata (14 kB)
Collecting mdurl~=0.1 (from markdown-it-py>=2.2.0->rich>=9.5.1->molecule->-r /etc/semaphore/requirements.txt (line 2))
  Downloading mdurl-0.1.2-py3-none-any.whl.metadata (1.6 kB)
Requirement already satisfied: cffi>=1.12 in /opt/semaphore/venv/lib/python3.11/site-packages (from cryptography->ansible-core>=2.12.10->molecule->-r /etc/semaphore/requirements.txt (line 2)) (1.16.0)
Requirement already satisfied: pycparser in /opt/semaphore/venv/lib/python3.11/site-packages (from cffi>=1.12->cryptography->ansible-core>=2.12.10->molecule->-r /etc/semaphore/requirements.txt (line 2)) (2.22)
Downloading hvac-1.2.0-py3-none-any.whl (149 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 149.9/149.9 kB 22.5 MB/s eta 0:00:00
Downloading molecule-24.2.1-py3-none-any.whl (113 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 114.0/114.0 kB 17.6 MB/s eta 0:00:00
Downloading redis-5.0.4-py3-none-any.whl (251 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 252.0/252.0 kB 38.7 MB/s eta 0:00:00
Downloading ara-1.7.1-py2.py3-none-any.whl (233 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 233.7/233.7 kB 37.3 MB/s eta 0:00:00
Downloading ansible_compat-24.6.0-py3-none-any.whl (23 kB)
Downloading click-8.1.7-py3-none-any.whl (97 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 97.9/97.9 kB 18.2 MB/s eta 0:00:00
Downloading click_help_colors-0.9.4-py3-none-any.whl (6.4 kB)
Downloading enrich-1.2.7-py3-none-any.whl (8.7 kB)
Downloading jsonschema-4.22.0-py3-none-any.whl (88 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 88.3/88.3 kB 16.6 MB/s eta 0:00:00
Downloading pbr-6.0.0-py2.py3-none-any.whl (107 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 107.5/107.5 kB 18.3 MB/s eta 0:00:00
Downloading pluggy-1.5.0-py3-none-any.whl (20 kB)
Downloading pyhcl-0.4.5-py3-none-any.whl (50 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 50.2/50.2 kB 9.7 MB/s eta 0:00:00
Downloading rich-13.7.1-py3-none-any.whl (240 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 240.7/240.7 kB 33.7 MB/s eta 0:00:00
Downloading wcmatch-8.5.2-py3-none-any.whl (39 kB)
Downloading cliff-4.7.0-py3-none-any.whl (81 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 81.2/81.2 kB 14.4 MB/s eta 0:00:00
Downloading attrs-23.2.0-py3-none-any.whl (60 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 60.8/60.8 kB 9.6 MB/s eta 0:00:00
Downloading autopage-0.5.2-py3-none-any.whl (30 kB)
Downloading bracex-2.4-py3-none-any.whl (11 kB)
Downloading cmd2-2.4.3-py3-none-any.whl (147 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 147.2/147.2 kB 26.4 MB/s eta 0:00:00
Downloading jsonschema_specifications-2023.12.1-py3-none-any.whl (18 kB)
Downloading markdown_it_py-3.0.0-py3-none-any.whl (87 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 87.5/87.5 kB 13.8 MB/s eta 0:00:00
Downloading prettytable-3.10.0-py3-none-any.whl (28 kB)
Downloading pygments-2.18.0-py3-none-any.whl (1.2 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.2/1.2 MB 65.5 MB/s eta 0:00:00
Downloading referencing-0.35.1-py3-none-any.whl (26 kB)
Downloading rpds_py-0.18.1-cp311-cp311-musllinux_1_2_x86_64.whl (1.3 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.3/1.3 MB 46.0 MB/s eta 0:00:00
Downloading stevedore-5.2.0-py3-none-any.whl (49 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 49.7/49.7 kB 7.6 MB/s eta 0:00:00
Downloading subprocess_tee-0.4.1-py3-none-any.whl (5.1 kB)
Downloading mdurl-0.1.2-py3-none-any.whl (10.0 kB)
Downloading wcwidth-0.2.13-py2.py3-none-any.whl (34 kB)
Building wheels for collected packages: pyperclip
  Building wheel for pyperclip (pyproject.toml): started
  Building wheel for pyperclip (pyproject.toml): finished with status 'done'
  Created wheel for pyperclip: filename=pyperclip-1.8.2-py3-none-any.whl size=11124 sha256=3675f8a744fdbacd1ddcde1949ba1d1c2c6c0d9728a09fc6ded7098954b35c0c
  Stored in directory: /home/semaphore/.cache/pip/wheels/70/bd/ba/8ae5c080c895c9360fe6e153acda2dee82527374467eae061b
Successfully built pyperclip
Installing collected packages: wcwidth, pyperclip, pyhcl, subprocess-tee, rpds-py, redis, pygments, PrettyTable, pluggy, pbr, mdurl, click, bracex, autopage, attrs, wcmatch, stevedore, referencing, markdown-it-py, hvac, cmd2, click-help-colors, rich, jsonschema-specifications, cliff, jsonschema, enrich, ara, ansible-compat, molecule
  WARNING: The script pygmentize is installed in '/home/semaphore/.local/bin' which is not on PATH.
  Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
  WARNING: The script pbr is installed in '/home/semaphore/.local/bin' which is not on PATH.
  Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
  WARNING: The script markdown-it is installed in '/home/semaphore/.local/bin' which is not on PATH.
  Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
  WARNING: The script jsonschema is installed in '/home/semaphore/.local/bin' which is not on PATH.
  Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
  WARNING: The scripts ara and ara-manage are installed in '/home/semaphore/.local/bin' which is not on PATH.
  Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
  WARNING: The script molecule is installed in '/home/semaphore/.local/bin' which is not on PATH.
  Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
Successfully installed PrettyTable-3.10.0 ansible-compat-24.6.0 ara-1.7.1 attrs-23.2.0 autopage-0.5.2 bracex-2.4 click-8.1.7 click-help-colors-0.9.4 cliff-4.7.0 cmd2-2.4.3 enrich-1.2.7 hvac-1.2.0 jsonschema-4.22.0 jsonschema-specifications-2023.12.1 markdown-it-py-3.0.0 mdurl-0.1.2 molecule-24.2.1 pbr-6.0.0 pluggy-1.5.0 pygments-2.18.0 pyhcl-0.4.5 pyperclip-1.8.2 redis-5.0.4 referencing-0.35.1 rich-13.7.1 rpds-py-0.18.1 stevedore-5.2.0 subprocess-tee-0.4.1 wcmatch-8.5.2 wcwidth-0.2.13
Starting semaphore server
Loading config
Validating config
Postgres ##########@####################:5432 ##########
Tmp Path (projects home) /tmp/semaphore
Semaphore v2.9.112-4f95ac8-1717065276
Interface
Port :3000
Server is running

time="2024-06-03T09:06:25Z" level=error msg="websocket: close sent" fields.level=Error
time="2024-06-03T09:06:25Z" level=error msg="close tcp <IP_ADDRESS>:3000-><IP_ADDRESS>:55060: use of closed network connection" fields.level=Error
time="2024-06-03T09:06:32Z" level=info msg="Task 98 added to queue"
time="2024-06-03T09:06:37Z" level=info msg="Set resource locker with TaskRunner 98"
time="2024-06-03T09:06:37Z" level=info msg="Task 98 removed from queue"
time="2024-06-03T09:06:40Z" level=info msg="Stopped running TaskRunner 98"
time="2024-06-03T09:06:40Z" level=info msg="Release resource locker with TaskRunner 98"
time="2024-06-03T09:13:20Z" level=info msg="Task 99 added to queue"
time="2024-06-03T09:13:22Z" level=info msg="Set resource locker with TaskRunner 99"
time="2024-06-03T09:13:22Z" level=info msg="Task 99 removed from queue"
time="2024-06-03T09:13:25Z" level=info msg="Stopped running TaskRunner 99"
time="2024-06-03T09:13:25Z" level=info msg="Release resource locker with TaskRunner 99"

time="2024-06-03T09:22:54Z" level=error msg="websocket: close sent" fields.level=Error
time="2024-06-03T09:22:54Z" level=error msg="close tcp <IP_ADDRESS>:3000-><IP_ADDRESS>:57488: use of closed network connection" fields.level=Error
time="2024-06-03T09:23:04Z" level=info msg="Task 100 added to queue"
time="2024-06-03T09:23:07Z" level=info msg="Set resource locker with TaskRunner 100"
time="2024-06-03T09:23:07Z" level=info msg="Task 100 removed from queue"
time="2024-06-03T09:23:10Z" level=info msg="Stopped running TaskRunner 100"
time="2024-06-03T09:23:10Z" level=info msg="Release resource locker with TaskRunner 100"
exit code: 0```

Web Run Logs:

```Task 98 added to queue
Started: 98
Run TaskRunner with template: Vault Test
Preparing: 98
installing static inventory
No collection/requirements.yml file found. Skip galaxy install process.
No collection/requirements.yml file found. Skip galaxy install process.
No role/requirements.yml file found. Skip galaxy install process.
No role/requirements.yml file found. Skip galaxy install process.
ansible-playbook [core 2.16.7]
  config file = /tmp/semaphore/ansible.cfg
  configured module search path = ['/tmp/semaphore/library']
  ansible python module location = /opt/semaphore/venv/lib/python3.11/site-packages/ansible
  ansible collection location = /tmp/semaphore/collections
  executable location = /opt/semaphore/venv/bin/ansible-playbook
  python version = 3.11.9 (main, Apr 14 2024, 13:40:00) [GCC 13.2.1 20231014] (/opt/semaphore/venv/bin/python3)
  jinja version = 3.1.4
  libyaml = True
Using /tmp/semaphore/ansible.cfg as config file
setting up inventory plugins
Loading collection ansible.builtin from
Parsed /tmp/semaphore/inventory_98.yml inventory source with yaml plugin
redirecting (type: callback) ansible.builtin.yaml to community.general.yaml
Loading collection community.general from /opt/semaphore/venv/lib/python3.11/site-packages/ansible_collections/community/general
redirecting (type: callback) ansible.builtin.yaml to community.general.yaml
Loading callback plugin community.general.yaml of type stdout, v2.0 from /opt/semaphore/venv/lib/python3.11/site-packages/ansible_collections/community/general/plugins/callback/yaml.py
[WARNING]: Skipping plugin (/home/semaphore/.local/lib/python3.11/site-
packages/ara/plugins/callback/ara_default.py), cannot load: No module named 'ara'
Skipping callback 'default', as we already have a stdout callback.
Skipping callback 'minimal', as we already have a stdout callback.
Skipping callback 'oneline', as we already have a stdout callback.```

### Manual installation - system information

_No response_

### Configuration

_No response_

### Additional information

_No response_

Same issue when running from the Web UI. Maybe it has something to do with all requirements.txt modules being installed in /home/semaphore and not in the venv
8a04df013a1c:~$ cat /etc/semaphore/requirements.txt
pywinrm
pymssql
netaddr
passlib

8a04df013a1c:~$ /opt/semaphore/venv/bin/pip3 list -v
Package            Version     Location                                            Installer
------------------ ----------- --------------------------------------------------- ---------
ansible            9.4.0       /opt/semaphore/venv/lib/python3.11/site-packages    pip
ansible-core       2.16.7      /opt/semaphore/venv/lib/python3.11/site-packages    pip
boto3              1.34.115    /opt/semaphore/venv/lib/python3.11/site-packages    pip
botocore           1.34.115    /opt/semaphore/venv/lib/python3.11/site-packages    pip
certifi            2024.2.2    /opt/semaphore/venv/lib/python3.11/site-packages    pip
cffi               1.16.0      /opt/semaphore/venv/lib/python3.11/site-packages    pip
charset-normalizer 3.3.2       /opt/semaphore/venv/lib/python3.11/site-packages    pip
cryptography       42.0.7      /opt/semaphore/venv/lib/python3.11/site-packages    pip
idna               3.7         /opt/semaphore/venv/lib/python3.11/site-packages    pip
Jinja2             3.1.4       /opt/semaphore/venv/lib/python3.11/site-packages    pip
jmespath           1.0.1       /opt/semaphore/venv/lib/python3.11/site-packages    pip
MarkupSafe         2.1.5       /opt/semaphore/venv/lib/python3.11/site-packages    pip
netaddr            1.3.0       /home/semaphore/.local/lib/python3.11/site-packages pip
packaging          23.2        /usr/lib/python3.11/site-packages
passlib            1.7.4       /home/semaphore/.local/lib/python3.11/site-packages pip
pip                24.0        /opt/semaphore/venv/lib/python3.11/site-packages    pip
pycparser          2.22        /opt/semaphore/venv/lib/python3.11/site-packages    pip
pymssql            2.3.0       /home/semaphore/.local/lib/python3.11/site-packages pip
pyparsing          3.1.1       /usr/lib/python3.11/site-packages
pyspnego           0.10.2      /home/semaphore/.local/lib/python3.11/site-packages pip
python-dateutil    2.9.0.post0 /opt/semaphore/venv/lib/python3.11/site-packages    pip
pywinrm            0.4.3       /home/semaphore/.local/lib/python3.11/site-packages pip
PyYAML             6.0.1       /opt/semaphore/venv/lib/python3.11/site-packages    pip
requests           2.32.3      /opt/semaphore/venv/lib/python3.11/site-packages    pip
requests-ntlm      1.2.0       /home/semaphore/.local/lib/python3.11/site-packages pip
resolvelib         1.0.1       /opt/semaphore/venv/lib/python3.11/site-packages    pip
s3transfer         0.10.1      /opt/semaphore/venv/lib/python3.11/site-packages    pip
setuptools         65.5.0      /opt/semaphore/venv/lib/python3.11/site-packages    pip
six                1.16.0      /opt/semaphore/venv/lib/python3.11/site-packages    pip
urllib3            2.2.1       /opt/semaphore/venv/lib/python3.11/site-packages    pip
xmltodict          0.13.0      /home/semaphore/.local/lib/python3.11/site-packages pip

The UI reports the error: msg: 'winrm or requests is not installed: No module named ''winrm'''
However, if I run the ansible-playbook command logged into the container

docker exec -it docker-semaphore-1 bash
/opt/semaphore/venv/bin/python3 /opt/semaphore/venv/bin/ansible-playbook -i inventory.yml --user svc-ansible --ask-pass --extra-vars '{"ANSIBLE_HOST_KEY_CHECKING":false,"ansible_connection":"winrm","ansible_port":"5986","ansible_winrm_server_cert_validation":"ignore","semaphore_vars":{"task_details":{"id":2,"url":null,"username":"admin"}}}' --fork 30 ping.yml

The above works OK and does not report the same module missing error
8a04df013a1c:~$ semaphore version
v2.9.112-4f95ac8-1717065276

A PR which removes the --user flag from the server/runner wrapper is incoming, I'm on it. Hopefully this will resolve the remaining issue related to requirements.txt

https://github.com/semaphoreui/semaphore/pull/2083

Just tried the current develop docker build - works now, thank you


Test Mention Bot, Just for test, don't merge

Coverage decreased (-1.2%) to 69.945% when pulling 7d38ca1e6a783bc64519c41e826f04d21f57e376 on asifdxtreme:master into 266f8224cf81755322aa61cdf956717888bc4059 on ServiceComb:master.

Codecov Report

Merging #101 into master will increase coverage by 0.07%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master     #101      +/-   ##
==========================================
+ Coverage   66.75%   66.83%   +0.07%
==========================================
  Files          18       18
  Lines        3956     3956
==========================================
+ Hits         2641     2644       +3
+ Misses       1150     1148       -2
+ Partials      165      164       -1

Impacted Files
Coverage Δ

server/service/services_dependency.go
67.6% <0%> (-0.71%)
:arrow_down:

server/service/instances.go
66.84% <0%> (+0.13%)
:arrow_up:

server/service/governservice.go
65.53% <0%> (+1.45%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 266f822...7d38ca1. Read the comment docs.

Coverage remained the same at 71.13% when pulling 7d38ca1e6a783bc64519c41e826f04d21f57e376 on asifdxtreme:master into 266f8224cf81755322aa61cdf956717888bc4059 on ServiceComb:master.

Coverage increased (+0.1%) to 71.231% when pulling 7d38ca1e6a783bc64519c41e826f04d21f57e376 on asifdxtreme:master into 266f8224cf81755322aa61cdf956717888bc4059 on ServiceComb:master.

Coverage remained the same at 71.13% when pulling 7d38ca1e6a783bc64519c41e826f04d21f57e376 on asifdxtreme:master into 266f8224cf81755322aa61cdf956717888bc4059 on ServiceComb:master.


Set ReplicaSet owner only if its a known kind

Fix display issue with non-standard controllers (e.g Argo)
This fixes the issue with the Ember UI in https://github.com/rancher/rancher/issues/31286

@dbason Can you backport this to release/v2.5 branch as well?

Sure thing, I'll do that now.


Uncaught TypeError: Cannot read property 'getGrammar' of undefined

[Enter steps to reproduce below:]

...
...

Atom Version: 1.0.2
System: Mac OS X 10.10.4
Thrown From: Hydrogen package, v0.4.2
Stack Trace
Uncaught TypeError: Cannot read property 'getGrammar' of undefined
At /Users/mark/.atom/packages/Hydrogen/lib/main.coffee:160

TypeError: Cannot read property 'getGrammar' of undefined
  at Object.module.exports.Hydrogen.run (/Users/mark/.atom/packages/Hydrogen/lib/main.coffee:160:25)
  at atom-text-editor.subscriptions.add.atom.commands.add.hydrogen:run (/Users/mark/.atom/packages/Hydrogen/lib/main.coffee:41:33)
  at CommandRegistry.module.exports.CommandRegistry.handleCommandEvent (/Applications/Atom.app/Contents/Resources/app.asar/src/command-registry.js:241:29)
  at /Applications/Atom.app/Contents/Resources/app.asar/src/command-registry.js:3:61
  at KeymapManager.module.exports.KeymapManager.dispatchCommandEvent (/Applications/Atom.app/Contents/Resources/app.asar/node_modules/atom-keymap/lib/keymap-manager.js:520:16)
  at KeymapManager.module.exports.KeymapManager.handleKeyboardEvent (/Applications/Atom.app/Contents/Resources/app.asar/node_modules/atom-keymap/lib/keymap-manager.js:355:22)
  at HTMLDocument.module.exports.WindowEventHandler.onKeydown (/Applications/Atom.app/Contents/Resources/app.asar/src/window-event-handler.js:177:20)

Commands
     -3:32.5.0 linter:set-bubble-transparent (atom-text-editor.editor.is-focused)
     -3:31.1.0 hydrogen:run (atom-text-editor.editor.is-focused)
     -3:29.4.0 core:move-down (atom-text-editor.editor.is-focused)
     -3:27.8.0 linter:set-bubble-transparent (atom-text-editor.editor.is-focused)
     -3:27.3.0 hydrogen:run (atom-text-editor.editor.is-focused)
     -3:22.4.0 core:backspace (atom-text-editor.editor.is-focused)
     -3:20.7.0 command-palette:toggle (atom-text-editor.editor.is-focused)
  2x -3:18.1.0 core:move-down (atom-text-editor.editor.mini.is-focused)
     -3:12.4.0 core:confirm (atom-text-editor.editor.mini.is-focused)
     -3:12.4.0 hydrogen:toggle-watches (atom-text-editor.editor)
     -2:54 editor:newline (atom-text-editor.editor.watch-input.is-focused)
     -2:53.2.0 core:backspace (atom-text-editor.editor.watch-input.is-focused)
     -2:44.4.0 hydrogen:run (atom-text-editor.editor.is-focused)
     -2:42.7.0 core:move-up (atom-text-editor.editor.is-focused)
  3x -2:38.7.0 core:move-down (atom-text-editor.editor.is-focused)
  6x -2:30.3.0 hydrogen:run (atom-text-editor.editor.is-focused)

Config
{
  "core": {
    "disabledPackages": [
      "linter-javac",
      "linter-pylint",
      "linter-ruby",
      "linter-scalac",
      "linter-chktex"
    ]
  },
  "Hydrogen": {}
}

Installed Packages
# User
Ensime, vundefined
Hydrogen, v0.4.2
autocomplete-bibtex, v0.5.7
language-arduino, v0.3.0
language-latex, v0.6.1
language-scala, v1.1.3
latex, v0.25.0
linter, v1.2.3
minimap, v4.11.2
minimap-git-diff, v4.1.6
scala-format, v0.1.8
tree-view-git-status, v0.1.1

# Dev
No dev packages

Did you figure out what was causing this?

Hi - I am not sure as I cannot reproduce it.
The error was caused when I accidentally had the settings tab open and in focus while trying to run the plugin with Hydrogen: Run.
But that seems to be a misbehavior in atom that is not reproducible.

OK, thanks!
On Sat, Jul 18, 2015, 9:22 AM Mark K<EMAIL_ADDRESS>wrote:

Hi - I am not sure as I cannot reproduce it.
The error was caused when I accidentally had the settings tab open and in
focus while trying to run the plugin with Hydrogen: Run.
But that seems to be a misbehavior in atom that is not reproducible.
—
Reply to this email directly or view it on GitHub
https://github.com/willwhitney/hydrogen/issues/83#issuecomment-122543485
.

Had the same error.
Steps to reproduce:

Open python file
Add few watches
Close python file (no open tabs left)
Run code in watches pane


importing duplicate templates causes backend to crash

if you add two templates with the same "name"
sqllite crashes
'template:default/docs-template' as full_name, 1 as generation, 'ae4dca91-4ec2-47a6-b971-eb33937d6851' as id, '20afeebe-f030-488e-ac5b-424f7c429861' as location_id - SQLITE_CONSTRAINT: UNIQUE constraint failed: entities.full_name
at CommonDatabase.transaction (webpack-internal:///../../plugins/catalog-backend/src/database/CommonDatabase.ts:84:15) type=plugin component=catalog-all-locations-refresh
expected behavior is it shouldn't be allowing me to add this template or had to use db primary id

That's not a crash though! It's just the database informing you that it works as intended - the catalog uses database constraints to strongly enforce that you don't end up in a weird state with duplicate identity keys. Entities are identified by the kind + namespace + name triplet; you can never have two with the same (case insensitive) triplet.
How did you add them? The catalog tries to stop you from making duplicates upfront (such as when registering new entities), but since these are files that can be edited and modified asynchronously by humans at any time in the future, all it can do is to tell you that there's an error and that you need to make adjustments.

That's not a crash though! It's just the database informing you that it works as intended - the catalog uses database constraints to strongly enforce that you don't end up in a weird state with duplicate identity keys. Entities are identified by the kind + namespace + name triplet; you can never have two with the same (case insensitive) triplet.
How did you add them? The catalog tries to stop you from making duplicates upfront (such as when registering new entities), but since these are files that can be edited and modified asynchronously by humans at any time in the future, all it can do is to tell you that there's an error and that you need to make adjustments.

I updated the api configuration yaml manually.
At least the frontend should have shown some kind of error message.

I updated the api configuration yaml manually.
At least the frontend should have shown some kind of error message.

Think we've arrived at this being a duplicate of #2876, so gonna close this and let's continue any discussion there.

Think we've arrived at this being a duplicate of #2876, so gonna close this and let's continue any discussion there.


TagBot trigger issue

This issue is used to trigger TagBot; feel free to unsubscribe.
If you haven't already, you should update your TagBot.yml to include issue comment triggers.
Please see this post on Discourse for instructions and more details.
If you'd like for me to do this for you, comment TagBot fix on this issue.
I'll open a PR within a few hours, please be patient!

Triggering TagBot for merged registry pull request: https://github.com/JuliaRegistries/General/pull/69196

Triggering TagBot for merged registry pull request
This extra notification is being sent because I expected a tag to exist by now, but it doesn't.
You may want to check your TagBot configuration to ensure that it's running, and if it is, check the logs to make sure that there are no errors.

Triggering TagBot for merged registry pull request: https://github.com/JuliaRegistries/General/pull/105363

Triggering TagBot for merged registry pull request
This extra notification is being sent because I expected a tag to exist by now, but it doesn't.
You may want to check your TagBot configuration to ensure that it's running, and if it is, check the logs to make sure that there are no errors.

Triggering TagBot for merged registry pull request
This extra notification is being sent because I expected a tag to exist by now, but it doesn't.
You may want to check your TagBot configuration to ensure that it's running, and if it is, check the logs to make sure that there are no errors.

Triggering TagBot for merged registry pull request
This extra notification is being sent because I expected a tag to exist by now, but it doesn't.
You may want to check your TagBot configuration to ensure that it's running, and if it is, check the logs to make sure that there are no errors.

Triggering TagBot for merged registry pull request
This extra notification is being sent because I expected a tag to exist by now, but it doesn't.
You may want to check your TagBot configuration to ensure that it's running, and if it is, check the logs to make sure that there are no errors.

Triggering TagBot for merged registry pull request: https://github.com/JuliaRegistries/General/pull/105427


Unable to use Python debugger

When I call the run_standard_evaluation function in my Python script, it outputs the following warning message and I am no longer able to use the Python debugger in my code due to sys.settrace() calls in the autoattack/checks.py file.

PYDEV DEBUGGER WARNING:
sys.settrace() should not be used when the debugger is being used.
This may cause the debugger to stop working correctly.
If this is needed, please check:
http://pydev.blogspot.com/2007/06/why-cant-pydev-debugger-work-with.html
to see how to restore the debug tracing back correctly.
Call Location:
File "/home/yuezuegu/.conda/envs/yuezuegu/lib/python3.8/site-packages/autoattack/checks.py", line 100, in check_dynamic
sys.settrace(tracefunc)

Hi,
check_dynamic is used to detect whether the model performs some kind of optimization during inference, but doesn't affect the attacks. Then, you might skip it to run the evaluation normally. I'm not sure whether there's a better solution, but please let me know in case you find it.

same problem！！！！！！Linux and Windows11！！！！  Pycharm and Vscode！！！！！！！！！！
when run_standard_evaluation is called. The debugger will stop

您好，您的邮件已收到!

@892446631 Please check the comment above. If that doesn't work for you, please provide more information about the issue.


🐛 Fix bug in building python component

A component that had a docsConfig attribute could not be built.

This makes it not error but also makes the docsConfig attribute not work


checksumFailMoreInfoUrl points to wrong anchor

VSCode Version: 1.3.0
OS Version: MacOS Mojave

Steps to Reproduce:

Corrupt your installation 🤷♂
Click More info button
URL points to https://code.visualstudio.com/docs/supporting/faq#_installation-appears-to-be-corrupt

Expected:
URL points to https://code.visualstudio.com/docs/supporting/faq#_installation-appears-to-be-corrupt-unsupported

Does this issue occur when all extensions are disabled?: Yes/No
Yes
Would have changed this myself but couldn't find where it comes from 🤔

👍 thanks!


[Feature request] Total number of anchors

This extension is very helpful, but the problem is that when you come back after a long time to a project and forget everything about the project, even if you have added any comment anchor or not.
In that case, it would be helpful to see a notification number, that how many total number of comment anchor we have.

I was thinking exactly the same thing. This would be really helpful for the same reasons that @mittalyashu mentions!

Hey,
As much as I would love to add this, it does not seem currently possible (See https://github.com/Microsoft/vscode/issues/62783). If this API were to be added, I'll implement support for it as soon as possible.
Thanks!

Thanks for the update @macjuui.
Can you keep this issue open so in future we are aware of this feature request?
And it will also avoid duplication of issue (features request)


"Logo.Image" seemingly non functional?

Despite using the same offsets and setup for the logo and even copy pasting the files for portal 1's logo over to test and I still have no logo displayed. This is especially strange considering the code to display this logo is very much present.
The sdk fork I'm using for my mod is mapbase in case that helps.
(apologies for the crowded screenshot I wanted to make sure you could see both the res and the files being present)
OS info
Processor:	Intel(R) Core(TM) i5-3340M CPU @ 2.70GHz   2.70 GHz
RAM:	        8.00 GB (7.87 GB usable)
Graphics:          Intel(R) HD Graphics 4000

ah that sucks. any possible workaround I could try?

Try entering in your mod title into gameinfo.txt, the logo's x/y position in GamepadUI s pulled from the calculations for m_LogoText for some reason, even when it's "empty" or nonexistent in gameinfo.

You do need to have a title set in gameinfo, the version included with HL2/P1 does the same thing. Just making it a single space works fine and doesn't mess with the logo image appearing in regular GameUI. Should probably look into fixing that


kubectl edit  fails with Edit cancelled, no changes made even when file has modifications

What keywords did you search in Kubernetes issues before filing this one? (If you have found any duplicates, you should instead reply there.):

edit ingress
edit cancelled

also Google search on error message and kubectls editing

Is this a BUG REPORT or FEATURE REQUEST? (choose one):
BUG REPORT

Kubernetes version (use kubectl version):
Client Version: version.Info{Major:"1", Minor:"7", GitVersion:"v1.7.4", GitCommit:"793658f2d7ca7f064d2bdf606519f9fe1229c381", GitTreeState:"clean", BuildDate:"2017-08-17T08:48:23Z", GoVersion:"go1.8.3", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"7", GitVersion:"v1.7.3", GitCommit:"2c2fe6e8278a5db2d15a013987b53968c743f2a1", GitTreeState:"clean", BuildDate:"2017-08-03T06:43:48Z", GoVersion:"go1.8.3", Compiler:"gc", Platform:"linux/amd64"}
Environment:

Cloud provider or hardware configuration:
GCE
OS (e.g. from /etc/os-release):

NAME="Ubuntu"
VERSION="16.04 LTS (Xenial Xerus)"

Kernel (e.g. uname -a):
Install tools:
Others:

What happened:

Open ingress file for editing
kubectl --namespace=<myspace> edit ingress <myingressname>
This opened up vim as expected. I made changes and saved them.
Close vim.
See message
"Edit cancelled, no changes made"
Tried this multiple times and made sure I always did a colon-w to write and same result everytime.
When I reopen the file, the changes have not been saved.

What you expected to happen:
Changes to ingress saved correctly.
How to reproduce it (as minimally and precisely as possible):
Seems like an intermittent problem (steps are detailed in the What happened section"), because I have been previous able to edit ingress entries correctly.
Anything else we need to know:

Can paste your manifest here?

hi @mengqiy - could you please clarify what you mean by manifest? the YAML file I used to create the ingress? I tried googling for kubernetes manifest, but could not find anything actionable.

could you please clarify what you mean by manifest?

Yes, yaml file
  tls:
  - hosts:
    - redacted.host.io
    secretName: somename-tls

to be
  tls:
   - secretName: somename-tls
     hosts:
      - redacted.host.io

There is no change between these two.
The order between hosts and secretName doesn't matter here, it's a map.
I think this is WAI. Closing.
Feel free to reopen if I miss something.

I have also faced same issue.
kubectl edit pod k8s-master-ag1gr1icp2m01 -n kube-system
error: pods "k8s-master-ag1gr1icp2m01" is invalid
error: pods "k8s-master-ag1gr1icp2m01" is invalid
A copy of your changes has been stored to "/tmp/kubectl-edit-oh43e.yaml"
error: Edit cancelled, no valid changes were saved.
Here is kubectl version
kubectl version
Client Version: version.Info{Major:"1", Minor:"9", GitVersion:"v1.9.1", GitCommit:"3a1c9449a956b6026f075fa3134ff92f7d55f812", GitTreeState:"clean", BuildDate:"2018-01-04T11:52:23Z", GoVersion:"go1.9.2", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"10", GitVersion:"v1.10.0+icp-ee", GitCommit:"9cb64de4ca4d039c35f4a29721aa5cf787648a15", GitTreeState:"clean", BuildDate:"2018-04-27T06:32:18Z", GoVersion:"go1.9.3", Compiler:"gc", Platform:"linux/amd64"}

I have similar issue.
error: pods "busybox-55f4f894ff-2pwkm" is invalid
A copy of your changes has been stored to "/tmp/kubectl-edit-b98rx.yaml"
error: Edit cancelled, no valid changes were saved.

Kubectl version:
[root@Ateiv-kubemaster ~]# kc version
Client Version: version.Info{Major:"1", Minor:"13", GitVersion:"v1.13.4", GitCommit:"c27b913fddd1a6c480c229191a087698aa92f0b1", GitTreeState:"clean", BuildDate:"2019-02-28T13:37:52Z", GoVersion:"go1.11.5", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"13", GitVersion:"v1.13.4", GitCommit:"c27b913fddd1a6c480c229191a087698aa92f0b1", GitTreeState:"clean", BuildDate:"2019-02-28T13:30:26Z", GoVersion:"go1.11.5", Compiler:"gc", Platform:"linux/amd64"}

Hey Guys!
Could you please check if you are using Tabs instead of spaces to indent the block or spaces instead of tabs?
You can also see the error when the file opens again for retry:
configmaps "coredns" was not valid:
* : Invalid value: "The edited file failed validation": [yaml: line 8: found a tab character where an indentation space is expected, invalid character 'a' looking for beginning of value]

I have also faced same issue.
[root@VM_2_105_centos tmp]# kubectl edit pods swift-b58c45f5c-5xlz4 -n kube-system
error: pods "swift-b58c45f5c-5xlz4" is invalid
A copy of your changes has been stored to "/tmp/kubectl-edit-1cec3.yaml"
error: Edit cancelled, no valid changes were saved.

Delete the file and re apply. Thanks to AWS.
kubectl delete -f https://raw.githubusercontent.com/kubernetes-sigs/aws-alb-ingress-controller/v1.1.8/docs/examples/alb-ingress-controller.yaml
kubectl apply -f https://raw.githubusercontent.com/kubernetes-sigs/aws-alb-ingress-controller/v1.1.8/docs/examples/alb-ingress-controller.yaml
Then edit and save like VI Editor.

This is happening probably because you have some mistake in the format of the changes you are making. Hence the changes are being rejected. Please double check that there is no tabs, spacing, extra char, format etc issue in the file changes you just made. Because that was the issue with me, and was getting the same error.

@i-Apoorva is spot on with this. I changed the image tag and noticed the file was auto-formatting on me.
containers:
- image: thing

became
containers:
  -  image: thing:1.1

fun little gotcha.

The funny thing is that I'm trying to edit the comments and it just doesn't see a change so I just get "Edit cancelled, no changes made."

I've had the same problem and the reason was the key I was try to configure was already somewhere else in the same file but I didn't notice it.

Just faced this issue. Note that after the first failure, it opens the file to edit again, when it does be careful to read it. The header is changed with an explanation on why it failed.

This is happening due to indentation. Once editing deployment file and changing image like nginx to nginx:1.16 vim editor is changing indentation with space. Remove the space it will work.


Add a tag 0.0.4 and update setup.py to use that

The version wasn't previously bumped for 0.0.2

@benpatterson @edx/devops

Have you happened to check what would happen if we tried to install a github hash with that tag? (i.e., will it uninstall v 0.1)?

@benpatterson In our jenkins env, we do pip --exists-action w so it rips out the old one and reinstalls.
I need/want to test on paver / AMI builds before merging, going to try and accomplish that before the cutoff tomorrow for the release.
I suspect we can also look at the jenkins worker logs on my build from platform?

Unfo you couldn't look at the logs on platform because the pip install is quiet; however, if you have devstack running you can just run the github.txt file and see if it uninstalls the other version...

@benpatterson I worked out my problems.  I accidentally tagged 0.0.3 onto the tip of master, which meant it wasn't checking out my bumped version in setup.py.
I tagged my branch as 0.0.4 and updated setup.py and platform accordingly and things are working better.

👍


fix(svelte): correct workspace version for svelte-query-devtools

Hey there! 👋🏻
When cloning the alpha branch, we cannot install dependencies because all Svelte packages use a no-longer-available version of @tanstack/svelte-query-devtools.
 ERR_PNPM_NO_MATCHING_VERSION_INSIDE_WORKSPACE  In examples/svelte/auto-refetching: No matching version found for<EMAIL_ADDRESS>inside the workspace

This PR corrects the version required by those packages.

Hi @RomainLanz , I actually fixed this in #5495, but thank you for pointing it out!


Fixes to dead links

fyi I tested increasing the timeout to a longer time to see if that helped (it did not), so I reduced the timeout time again.


commit cargo.lock file

Could you commit the cargo.lock file ? I'm looking to package this for nixos.

Acknowledged. I will add one and publish a new version.

Done, will publish in new bersion


Proper cleanup of resources/folders during shutdown and startup

staging canaries were failing because one of the hosts ran out of disk space. I checked and it was a folder under /tmp/bacalhau-results1643036303 that was taking the whole space. We need to make sure we are properly cleaning all of our resources during both shutdown, and startup in case of ungraceful shutdown

Thanks Ross for picking this up. Though there are still areas where we don't cleanup the resources properly, including:

Job failure or cancellation
Result rejection by requester node
No response from requester node to approve or reject the proposal. Though this requires implementing timeouts in the compute nodes on how long they should wait, which we don't have today. Something similar to Requester's Housekeeping
Cleaning up during startup to handle ungraceful shutdowns
Lastly cleaning up of inputs as the fix is only cleaning outputs

I've added the bullet points remaining to the initial ticket descriptions as a TODO checklist, hope that's okay.


mean subtraction and rgb to bgr in preprocessing step in inception models

This is preprocessing function of your inception v4.
def preprocess_input(x):
    x = np.divide(x, 255.0)
    x = np.subtract(x, 0.5)
    x = np.multiply(x, 2.0)
    return x

why It is totally different from other models preprocessing?
1. Why there is no mean subtraction?
2. Why there is no RGB to BGR? Instead you used RGB
3. Mapping between [-1,1] or [-x, +x] is normal for all inception models?
This is preprocessing function of VGG and ResNet in Keras:
def preprocess_input(x, data_format=None):
    if data_format is None:
        data_format = K.image_data_format()
    assert data_format in {'channels_last', 'channels_first'}

    if data_format == 'channels_first':
        # 'RGB'->'BGR'
        x = x[:, ::-1, :, :]
        # Zero-center by mean pixel

        x[:, 0, :, :] -= 103.939
        x[:, 1, :, :] -= 116.779
        x[:, 2, :, :] -= 123.68
    else:
        # 'RGB'->'BGR'
        x = x[:, :, :, ::-1]
        # Zero-center by mean pixel
        x[:, :, :, 0] -= 103.939
        x[:, :, :, 1] -= 116.779
        x[:, :, :, 2] -= 123.68
    return x

Also Caffe models use mean subtraction and RGB to BGR.

Hi @i-akbari
You are comparing apple's to oranges.
I would encourage you to read Google's implementation of the Inception architectures all of which use the same preprocessing scheme: https://github.com/tensorflow/models/blob/master/slim/preprocessing/inception_preprocessing.py#L237-L275

The Inception architecture does not require it, therefore it is not done.
The original weights trained in-house at Google used an RGB scheme. If you want to retrain the network from scratch, using RGB or BGR are both acceptable options. Consistency in choice between training and testing is all that matters.
Yes.

Thanks @kentsommer


Access application closed reason as bytes/string

There is the error variant ConnectionError::ApplicationClosed(ApplicationClose), where ApplicationClose is
#[derive(Debug)]
pub struct ApplicationClose {
    code: VarInt,
    reason: Box<[u8]>,
}

But there is no way to access this reason directly, neither as a &[u8] nor as a String. This seems like an oversight, since accessing the application-layer disconnection reason is very useful for e.g. user-friendly disconnection messages. The only way to get this reason out right now is via the Display impl, which also adds some extra info like the error code, which I don't want.
There should be API like
fn code(&self) -> VarInt;

fn reason(&self) -> &[u8];

to access the components, or just make the fields of ApplicationClose public. Why do they have to be private anyway?

I missed that, Thank you


Crash on pressing NUMPAD 8

Placebo 7.3.3 crashes when I try to type a seed including an 8 (used NUMPAD). Based on the commits I assume 9 will also crash. Going back to the previous version solves the issue.
EDIT Confirmed it's just NUMPAD 8/9 keys pressed at any point in the game. They are bound to patreon trails.
debug.log

confirming as well, reported by Matrixis while playing ftb genesis but num pad 9 for him
https://gist.github.com/Saereth/be3192ba61c2100830a009379b3eaeb9

Numpad 9, but same bug https://mclo.gs/oAXgHVB
Encountered while playing ATM8. Issue follows patreon bindings and occurs in menu or while typing connection info. Does not occur once connected to world.


WIP: Set up spack environment to handle build

Description
The current build instructions are a bit ad-hoc. I have attempted to make the build process more robust using Spack. In particular, I have added a spack.yaml file which defines a custom Spack environment to be used for development.
Addressed (at least partially) #3, #20, relates to #86.
Type of change

[x] Build system improvements
[x] Requires documentation updates

Testing
The build was executed and confirmed to work on my laptop. The resulting test binaries which were built were run and all passed.
Test Configuration:

OS: Linux Mint 21 (Ubuntu 22.04)
SYCL implementation: HipSYCL 0.9.2
MPI details: MPICH 4.0.2
Hardware: Intel Icelake

Checklist:

[x] I have added tests that prove my fix is effective or that my feature works
[x] New and existing unit tests pass locally with my changes
[x] Any new dependencies are automatically built for users via cmake
[x] I have used understandable variable names
[x] I have run clang-format against my *.hpp and *.cpp changes
[x] I have run cmake-format against my changes to CMakeLists.txt
[x] I have run black against changes to *.py
[x] I have made corresponding changes to the documentation
[x] I have performed a self-review of my own code
[x] I have commented my code, particularly in hard-to-understand areas
[x] My changes generate no new warnings

@jwscook @will-saunders-ukaea This is in a state I'm reasonably happy with. It is ready for review and can be merged after NESO-Spack#3.


Refactoring?

One of the cool ideas brought up (by @md678685) was making the code more modular.
I think this would really make our lives a lot easier, and hopefully encourage people to hack on different parts... I see two options:

Make a different repo for each component. The core repo would contain the restarting/updating code. There would also be a separate repo with shared code/libraries (such as github api stuff). Then, all of the other subsystems can live in their own repo. Upside: Cleaner organization and enforces modularity of components. Downside: More complex to manage starting/stopping components and coordinating PRs with multi-component parts might require some engineering (which I think might be solved by a issue command such as @chaosbot depends on #Chaosthebot/Chaos#600 or something).

Use a strict well-designed directory/module hierarchy and enforce by conventions. (I personally lean here)

I lean 2 as well. Multiple repos would be super interesting but might prove to be complicated to orchestrate. I'm interested in others thoughts on this.

My idea is that we break the code into plugins (Some options for how we do that are here: http://wehart.blogspot.com/2009/01/python-plugin-frameworks.html). The idea being that plugins are sandboxed so that if an error occurs and a plugin crashes, it won't kill the core code, and could possibly be immediately restarted.

That's assuming all the modules will be in python though. I'd be interested in making a node module.
Maybe we can have different central language hubs which connect to a central controller.

And how about multi modules living into the same repository ?

I don't think I was the first to suggest modularity :p
I was thinking that the "core" would be the "central controller", as suggested by @Redmega, and would launch components either as child processes or daemons and have a common protocol to interact with it from modules (perhaps through a Unix/TCP socket), as suggested by @mpnordland.
Each component would be in a separate repo, and would contain a metadata file with information about how they should be launched. Repos could become recognised as components to launch by through a config for the core, and if new components are added, a PR is submitted to add them.
Shared libraries could be independent repos which are referenced by submodules, which would ease synchronising the libraries between repos and should still allow for some components to stay on older versions of shared libraries if breaking changes are introduced through library updates until the component itself is updated.
I don't think multiple separate modules should live in the same repository - the separation of modules is what will help identify separate modules.

😊

@mark-i-m Any particular reason for closing?

I didn't realize there was active work still going on, as the repo has been silent for a while TMK... Reopening

I mean, it's more that if some work does happen then this will be much easier to find.

I would like to close this PR to clean up my dashboard. Does anyone have objections?

Go for it. Work has left me not able to contribute to Chaos anymore.

No objection here.


feat(compass): add filter to get all types and move gettypes to postgres

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

69 of 83 (83.13%)  changed or added relevant lines in 4 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage increased (+0.07%) to 81.238%

Changes Missing Coverage
Covered Lines
Changed/Added Lines
%

internal/server/v1beta1/type.go
13
17
76.47%

internal/store/postgres/asset_repository.go
27
37
72.97%

Totals

Change from base Build<PHONE_NUMBER>:
0.07%

Covered Lines:
4529

Relevant Lines:
5575

💛  - Coveralls

@StewartJingga yes that should be fine


(fix) Date Validation Error Messages

Fixes the error message when date is invalid
Also fixes binding variables to error string

[error] missing Gettext bindings: [:after] (backend NetAdminWeb.Gettext, locale "en", domain "errors", msgctxt nil, msgid "should be after %{after}.")

BEFORE

AFTER

Happy new year 🍾

Hey @kyle-neal , Thanks for the pull request ❤ and sorry for the late reply.
I'm using the same format as Ecto validation which uses %{} for placeholders, e.g. https://github.com/elixir-ecto/ecto/blob/v3.9.4/lib/ecto/changeset.ex#L2234 so that it can work with gettext translation. Using #{} will make it replace the string in place making translation impossible.
You might need to pass the validation through gettext so that placeholders are properly replaced with correct validation values.


Bump channel/rubocop-0-63 to use rubocop 0.63.1

RuboCop 0.63.1 has been released.
https://github.com/rubocop-hq/rubocop/releases/tag/v0.63.1
This PR performed the following steps with reference to #128.
git checkout channel/rubocop-0-63
make image
git checkout -b update_0_63_1
docker run --user root --rm --volume "$(pwd)":/usr/src/app --workdir
/usr/src/app codeclimate/codeclimate-rubocop bundle update rubocop
make docs
make test

@koic thank you! the channel was bumped on a separate PR. Let us know if anything


[WIP] Xml interpolation expr improvement

Purpose
Update XML interpolation syntax from {{expr}} to ${expr}
//xml x = xml `<Person name={{name}}>{{content}}</Person>`;
xml x = xml `<Person name=${name}>${content}</Person>`;

Remove the ability to create XML elements types and attributes by interpolating tag name
\\ We can no longer do below
xml x = xml `<${elemName}>content</${elemName}>`;
xml y = xml `<elem ${attName}="value" />`;

Fixes https://github.com/ballerina-platform/ballerina-lang/issues/13946 and https://github.com/ballerina-platform/ballerina-lang/issues/13718
Also fixes https://github.com/ballerina-platform/ballerina-lang/issues/10935

Codecov Report

:exclamation: No coverage uploaded for pull request base (master@a9d105d). Click here to learn what that means.
The diff coverage is 100%.

@@            Coverage Diff            @@
##             master   #14281   +/-   ##
=========================================
  Coverage          ?   64.79%
  Complexity        ?      425
=========================================
  Files             ?     2182
  Lines             ?    96303
  Branches          ?    12848
=========================================
  Hits              ?    62400
  Misses            ?    28865
  Partials          ?     5038

Impacted Files
Coverage Δ
Complexity Δ

.../java/org/ballerinalang/model/values/BXMLItem.java
72.34% <100%> (ø)
0 <0> (?)

...erinalang/compiler/parser/BLangParserListener.java
70.44% <100%> (ø)
0 <0> (?)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update a9d105d...5180961. Read the comment docs.


Test/run source-code within OR Simulation

Is your feature request related to a problem? Please describe.

for the ev3 (and other systems), it would be nice to have the ability to run modify the source code (c, python, java...) within the simulator

Describe the solution you'd like

Include a additonal button "Sim" which open the OR Simulation

Unfortunately this is impossible

why?

the generator of code for the simulation takes the blockly XML and translates it to some byte code, that is executed by the byte code interpreter of the simulation. Our simulation has no knowledge about the target language of the real robot (Python + Java for ev3, C++ for Calliope). If we get Python code edited by the user, for instance, what should we do? Program a full Python interpreter? Get Python running in the Javascript engine of a browser? the same for Java and C++? This is impossible for the team.