Adding a few rof notebooks

Initial commits for ROF notebooks.
An ultimate set of the notebooks intend to mimic old ROF diagnostic plots
This PR is just starting with a few notebooks.
All Submissions:

[x] Have you followed the guidelines in our Contributor's Guide (including the pre-commit check)?
[x] Have you checked to ensure there aren't other open Pull Requests for the same update/change?

New Feature Submissions:

[x] Does your submission pass tests?
[x] Have you lint your code locally prior to submission?

Changes to Core Features:

[x] Have you added an explanation of what your changes do and why you'd like us to include them?
[ ] Have you successfully tested your changes locally?

Hey @nmizukami ! Thanks so much for adding these notebooks! I made those changes we discussed to cupid-run and the config file to include rof in running cupid and in the jupyter book table of contents.
One note: I think we'll need to provide math in cupid-analysis if you prefer to use that for sqrt rather than numpy in metrics.py? I don't think this should be a problem, but just let me know if you'd like me to do that?
Would you be able to pull these changes in locally, test running your notebook with cupid-run and make sure that things look as you expect?

Hi Teagan (@TeaganKing), The notebook almost ran with cupid-run -rof. One error was reading geopackage file (gis vector data) via geopanda.
You can see /glade/work/mizukami/CUPiD/examples/coupled_model/month_annual_flow.ipynb.
cupid prints on screen this:
RROR 1: PROJ: proj_create_from_database: /glade/u/apps/casper/23.10/spack/opt/spack/proj/8.2.1/gcc/12.2.0/7gif/share/proj/proj.db contains DATABASE.LAYOUT.VERSION.MINOR = 2 whereas a number >= 3 is expected. It comes from another PROJ installation.

I have seen this before. I don't fully understand the error, but this is coming from pyogrio package that came with geopandas. pyogrio is used internally in geopandas.
When I ran the notebook outside cupid, it runs fine but I activated Python [conda-env:cupid-analysis] environment in the jupyterhub. I see another one called cupid-analysis, which I believe cupid actually uses. I saw similar error when I use cupid-analysis. wondering what is the difference between Python [conda-env:cupid-analysis] and cupid-analysis?
hopefully I am simply setting something e.g., environment incorrectly...

Hi @nmizukami , Sorry I let this slip! In the environment in which this was working, did you have a particular version of geopandas or pyogrio pinned? I could also add that to the environment yaml specification. Or when you previously ran into this error, did you have another solution?
This error may be because PROJ is already installed-- I'm not sure where at this point, but can look into that.

It sounds like there may be some issue related to the ipykernel installation. I think one of these might be the installation from ipykernel (a soft linked conda environment) and the other may be a conda environment found elsewhere (possibly an outdated cupid-analysis that doesn't include geopandas?).
I had updated a test environment but not my actual cupid-analysis environment; I'm doing that now and will test your notebook out. This is probably not the most efficient workflow, but I wonder if it might also be worth removing your cupid-analysis environment, see if it's still listed as an option in JupyterHub, make sure that both versions are removed, and then re-install a clean version?

I did the following steps to remove cupid-analysis env and reinstall it on terminal.
mamba remove --name cupid-analysis --all
mamba env create -f environments/cupid-analysis.yml

It did not fix it. After removing cupid-analysis, jupyterhub still showed cupid-analysis, though [conda-env:cupid-analysis] was gone.

Hi @TeaganKing , trying to run conda list to see what packages are there in cupid-analysis env when running cupid-run. So unfortunately including conda list in the notebook cause error in cupid run:
SyntaxError: An error happened when checking the source code.
:25:7: invalid syntax

conda list

casper-login1:/glade/work/mizukami/CUPiD/examples/coupled_model (main_adding_rof)> cupid-run -rof
/glade/work/mizukami/conda-envs/cupid-dev/lib/python3.11/site-packages/ploomber/dag/dag.py:455: UserWarning:
========================================================================================= DAG render with warnings =========================================================================================
----------------------------------------------------------------- NotebookRunner: index -> File('computed_notebook...ucture/index.ipynb') ------------------------------------------------------------------
----------------------------------------------------------------- /glade/work/mizukami/CUPiD/examples/nblibrary/infrastructure/index.ipynb -----------------------------------------------------------------
These parameters are not used in the task's source code: 'CESM_output_dir', 'lc_kwargs', 'serial', and 'subset_kwargs'
----------------------------------------------------------- NotebookRunner: month_annual_flow -> File('computed_notebook..._annual_flow.ipynb') ------------------------------------------------------------
---------------------------------------------------------------- /glade/work/mizukami/CUPiD/examples/nblibrary/rof/month_annual_flow.ipynb -----------------------------------------------------------------
These parameters are not used in the task's source code: 'CESM_output_dir', 'lc_kwargs', 'serial', and 'subset_kwargs'
============================================================================================ Summary (2 tasks) =============================================================================================
NotebookRunner: index -> File('computed_notebook...ucture/index.ipynb')
NotebookRunner: month_annual_flow -> File('computed_notebook..._annual_flow.ipynb')
========================================================================================= DAG render with warnings =========================================================================================

  warnings.warn(str(warnings_))
Executing: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 3/3 [00:02<00:00,  1.03cell/s]
Building task 'month_annual_flow':  50%|███████████████████████████████████████████████████████████████████                                                                   | 1/2 [00:02<00:02,  2.92s/itERROR 1: PROJ: proj_create_from_database: /glade/u/apps/casper/23.10/spack/opt/spack/proj/8.2.1/gcc/12.2.0/7gif/share/proj/proj.db contains DATABASE.LAYOUT.VERSION.MINOR = 2 whereas a number >= 3 is expected. It comes from another PROJ installation.
                                                                                /glade/u/apps/opt/conda/condabin/conda                                                                                | 5/69 [00:20<03:39,  3.44s/cell]
Executing:  90%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████▍               | 62/69 [03:53<00:26,  3.76s/cell]
Building task 'month_annual_flow': 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 2/2 [03:56<00:00, 118.06s/it]
Traceback (most recent call last):
  File "/glade/work/mizukami/conda-envs/cupid-dev/bin/cupid-run", line 8, in <module>
    sys.exit(run())
             ^^^^^
  File "/glade/work/mizukami/conda-envs/cupid-dev/lib/python3.11/site-packages/click/core.py", line 1157, in __call__
    return self.main(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/glade/work/mizukami/conda-envs/cupid-dev/lib/python3.11/site-packages/click/core.py", line 1078, in main
    rv = self.invoke(ctx)
         ^^^^^^^^^^^^^^^^
  File "/glade/work/mizukami/conda-envs/cupid-dev/lib/python3.11/site-packages/click/core.py", line 1434, in invoke
    return ctx.invoke(self.callback, **ctx.params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/glade/work/mizukami/conda-envs/cupid-dev/lib/python3.11/site-packages/click/core.py", line 783, in invoke
    return __callback(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/glade/work/mizukami/CUPiD/cupid/run.py", line 290, in run
    dag.build()
  File "/glade/work/mizukami/conda-envs/cupid-dev/lib/python3.11/site-packages/ploomber/dag/dag.py", line 557, in build
    report = callable_()
             ^^^^^^^^^^^
  File "/glade/work/mizukami/conda-envs/cupid-dev/lib/python3.11/site-packages/ploomber/dag/dag.py", line 662, in _build
    raise build_exception
  File "/glade/work/mizukami/conda-envs/cupid-dev/lib/python3.11/site-packages/ploomber/dag/dag.py", line 591, in _build
    task_reports = self._executor(dag=self, show_progress=show_progress)
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/glade/work/mizukami/conda-envs/cupid-dev/lib/python3.11/site-packages/ploomber/executors/serial.py", line 203, in __call__
    raise DAGBuildError(str(exceptions_all))
ploomber.exceptions.DAGBuildError:
============================================================================================= DAG build failed =============================================================================================
----------------------------------------------------------- NotebookRunner: month_annual_flow -> File('computed_notebook..._annual_flow.ipynb') ------------------------------------------------------------
---------------------------------------------------------------- /glade/work/mizukami/CUPiD/examples/nblibrary/rof/month_annual_flow.ipynb -----------------------------------------------------------------
---------------------------------------------------------------------------
Exception encountered at "In [24]":
---------------------------------------------------------------------------
NameError                                 Traceback (most recent call last)
Cell In[24], line 2
      1 column_stat = []
----> 2 gauge_shp_all_case = gauge_shp.copy(deep=True)
      3 for case, grid_name in cases.items():
      4     gauge_shp_all_case = gauge_shp_all_case.merge(
      5         gauge_shp1[case][["id", f"{error_metric}_{grid_name}"]],
      6         left_on="id",
      7         right_on="id",
      8     )

NameError: name 'gauge_shp' is not defined

ploomber.exceptions.TaskBuildError: Error when executing task 'month_annual_flow'. Partially executed notebook available at /glade/work/mizukami/CUPiD/examples/coupled_model/computed_notebooks/quick-run/rof/month_annual_flow.ipynb
ploomber.exceptions.TaskBuildError: Error building task "month_annual_flow"
============================================================================================= Summary (1 task) =============================================================================================
NotebookRunner: month_annual_flow -> File('computed_notebook..._annual_flow.ipynb')
============================================================================================= DAG build failed =============================================================================================

Hi @TeaganKing,
Small good new is that I got it run without the geopanda error. The trick is to add this
os.environ['PROJ_LIB']='/glade/work/mizukami/conda-envs/cupid-analysis/share/proj'
before loading geopandas.
However, I don't think this is permanent solution. I still try to consult with CISL.
I was able to create /glade/work/mizukami/CUPiD/examples/coupled_model/computed_notebooks/quick-run/_build/html/
How do you usually open under HPC. I was trying to open firefox in derecho/casper, but it is very slow. Wonder if there is any other ways to look.

Hi @nmizukami , I'm glad that is temporarily working (but of course we need this to work for any user's environment). Yes, I think this would be a good conversation to have with CISL.
Regarding looking at output, see the second section on this page for recommendations on NCAR machines.

Hey @nmizukami , I added a PR to bring rof into run.py.

To-do:

[ ] update readme to include 'rof' on line 104: -rof, --river-runoff Run river runoff component diagnostics

Per conversation with @nmizukami , this is now working when he cloned the cupid package, installed both of the python environments, and ran create_conda_kernel under cupid-analysis, instead of python -m ipykernel install --user --name=cupid-analysis.
I also updated the readme to include rof and fixed a number of issues from pre-commit checks.
Could you please fix these two errors from pre-commit checks in examples/nblibrary/rof/scripts/metrics.py? If metrics.py is not currently being used, it would also be great to either 1) remove this script and add it in when the notebooks that are going to use it are included in CUPiD, OR 2) fix the scripts to make sure they work and include them in this PR.
examples/nblibrary/rof/scripts/metrics.py:363:26: F821 undefined name 'myCount'
examples/nblibrary/rof/scripts/metrics.py:424:26: F821 undefined name 'myCount'

After updating my conda environment, I ran into the following rof-related errors. I'll plan to look into these issues in more detail once jupyterhub is back up. It seems like they could be a permission-related error where the dictionaries are not being properly populated and thus the keys are incorrect.
--------------------------------------------------------------------------- NotebookRunner: month_annual_flow -> File('computed_notebook..._annual_flow.ipynb') ----------------------------------------------------------------------------
-------------------------------------------------------------- /glade/work/tking/cupid_project/other_cupids/naoki_cupid/CUPiD/examples/nblibrary/rof/month_annual_flow.ipynb ---------------------------------------------------------------
---------------------------------------------------------------------------
Exception encountered at "In [12]":
---------------------------------------------------------------------------
KeyError                                  Traceback (most recent call last)
Cell In[12], line 6
      3 gauge_plot[gname] = {}
      4 for case, _ in cases.items():
      5     gauge_ix, seg_index, seg_id = find_seg_id(
----> 6         site_id, reachID[case], gauge_reach_lnk[case]
      7     )
      8     gauge_plot[gname][case] = [gauge_ix, seg_index, seg_id]

KeyError: 'f09_f09_mg17_mosart'

ploomber.exceptions.TaskBuildError: Error when executing task 'month_annual_flow'. Partially executed notebook available at /glade/work/tking/cupid_project/other_cupids/naoki_cupid/CUPiD/examples/coupled_model/computed_notebooks/quick-run/rof/month_annual_flow.ipynb
ploomber.exceptions.TaskBuildError: Error building task "month_annual_flow"
---------------------------------------------------------------------------- NotebookRunner: ocean_discharge -> File('computed_notebook...an_discharge.ipynb') -----------------------------------------------------------------------------
--------------------------------------------------------------- /glade/work/tking/cupid_project/other_cupids/naoki_cupid/CUPiD/examples/nblibrary/rof/ocean_discharge.ipynb ----------------------------------------------------------------
---------------------------------------------------------------------------
Exception encountered at "In [16]":
---------------------------------------------------------------------------
KeyError                                  Traceback (most recent call last)
Cell In[16], line 34
     32     id_list = gauge_shp1[case]["route_id"].values
     33 else:
---> 34     id_list = gauge_shp1[case][gauge_shp1[case]["ocean"] == ocean_name][
     35         "route_id"
     36     ].values
     38 reach_index = get_index_array(reachID[case], id_list)
     39 seas_data_vector = seas_data[case][q_name].stack(seg=("lat", "lon"))

KeyError: 'f09_f09_mg17_mosart'

did I mess this up?  I was rebasing NCAR/main and also nmizukami/main_adding_rof to get my local to up-to-date.

Thanks for fixing that! These are now running smoothly on my end with those updated permissions!

In month_annual_flow.ipynb, I also have a few additional comments (but the file was too large to render/comment on particular lines):
All sections: Can you please remove empty cells? I think the ‘go back to top’ sections could be removed if this is being run all at once?

Removed empty cells and removed ‘go back to top’ link.

Section 1. Are the lists of cases intended to be regularly used options? Is the yaml file that’s loaded going to be consistent, or is it intended to be updated by user?

Yes, case is CESM case, which CESM modeler put and this has to be provided by the user. I tried adding some comments what need to be provided here.  Multiple cases are allowed to compare the results from different cases

2.1 – there’s a typo ‘Mmonthly’

Fixed.

2.6 - can you add a docstring to define what’s happening in this function?

This function is removed.

3.1 – I think this cell may have an empty slice error.

I think I know how this happens. I think this is due to the inconsistent observation data length across the sites (some sites provide the data for a short period). No observed flow data is available depending analysis (or simulation) periods. Can I fix this in another PR?

I started to look at this, and I have a lot of questions and suggestions -- @nmizukami and @TeaganKing could we try to find a time early next week to meet? I'd like to discuss a few things that might be tough to squeeze into an in-line review on this PR. Some initial comments, though (and maybe these changes will make it a little easier to go back through line-by-line asynchronously):

It looks like this is failing the pre-commit style checks
I noticed in month_annual_flow.ipynb you have a logical flag parallel that enables using PBSCluster when set to true. If you look at examples/nblibrary/ocn/ocean_surface.ipynb you'll see how CUPiD already passes a serial flag and uses a LocalCluster when that is set to false... I haven't looked at the other runoff notebooks, but we need to avoid casper- or derecho-specific blocks of code
The notebooks in examples/nblibrary/rof should not have any output in them

Hi Mike (@mnlevy1981) and Teagan (@TeaganKing), yes, I can meet on Monday or Tuesday (my calendar is up-to date). I was wondering about PBSCluster (2nd point)

To-do

[ ] replaced PBSCluster with LocalCluster, then follow the example in ocn notebook (use serial logical flag)
[ ] activated cupid-dev and then Installed pre-commit under CUPiD/.git/hook.
[ ] clear outputs from all the cells.
[ ] add key_metrics/config.yml and modify case:grid input.

Updated key_metrics/config.yml  and coupled_model/config.yml for rof
modify two notebooks based on config changes so now they run.
Review is needed and some science questions came up (e.g., what to do if you plot for time period when no observation is available. Are the other notebooks comparing the model outputs with observations??)

Hey @nmizukami , thanks for these updates.
Not all notebooks are comparing with observations, but you can see an example of an observational comparison in the glacier notebook & corresponding config.yml details.
I think that if you are plotting for a time period where observations are not available, perhaps a warning statement that the obs are unavailable would be useful?

And I'll review after our discussion on Thursday.

Right now I am pointing to case /glade/campaign/cesm/development/cross-wg/diagnostic_framework/CESM_output_for_testing/b.e23_alpha16b.BLT1850.ne30_t232.054
The time period for this case is year 0001-0102, when for sure there is no observation for any components. So I thought this config is meant to compare the simulation with some base simulation, to see the model comparison or something like that, not meant to validate the model component with observations.
Just wanted to understand the context of this setup. just with current config, the rof notebooks look less interesting, but technically the notebook works now (I believe).
If config point to any CESM cases that use 20th-21st century, rof notebook automatically adds the observed streamflow to the plots, and compare the simulations with observations.

The key setup here that's different from the coupled-model example is in the 'global params' section of the config file, where we have both a case name for the case you're looking at, as well as the base_case_name for a comparison case. The observations are defined separately in each individual notebook config section at this point.

That sounds good that plots are generated without obs if obs do not exist.

@nmizukami is planning to do the following:

[ ] implement comparison with base_case in addition to obs
[ ] remove years that overwrite config file start/end years
[ ] include analysis-period configuration parameter to specify e.g. 10 years so that users don't need to run 100 years unless they really want to do so.
[ ] test cupid-run from key_metrics directory

Once these items are done, @TeaganKing can review.

@nmizukami is planning to do the following:

[ ]  implement comparison with base_case in addition to obs
[ ]  remove years that overwrite config file start/end years
[ ]  include analysis-period configuration parameter to specify e.g. 10 years so that users don't need to run 100 years unless they really want to do so.
[ ]  test cupid-run from key_metrics directory

Once these items are done, @TeaganKing can review.

Hi @TeaganKing, all are done!

It looks like the notebooks ran in key_metrics in about 15 minutes. Both notebooks seem to be running smoothly. One additional comment regarding the figures-- in month_annual_flow.ipynb, there are a number of figures that are saved but not directly plotted in the notebook. For the purposes of generating the juptyerbook, I think it'd be best to show the plots directly in the notebook.
In the coupled_model example, in Cell 5, there was an error with no files to open. It may be best to leave the key_metrics example, and take out rof from the coupled_model example since we'll need to update that configuration file anyways and create an 'additional metrics' example.

Hey @nmizukami , thanks for all your work on this! It's definitely getting much closer!
I made a few in-line comments above and also wanted to remind you of @mnlevy1981 's comment from above: "I noticed in month_annual_flow.ipynb you have a logical flag parallel that enables using PBSCluster when set to true. If you look at examples/nblibrary/ocn/ocean_surface.ipynb you'll see how CUPiD already passes a serial flag and uses a LocalCluster when that is set to false... I haven't looked at the other runoff notebooks, but we need to avoid casper- or derecho-specific blocks of code". This same structure is also currently used in ocean_discharge.ipynb. You can see details on the current serial vs parallel implementation in our documentation.

I believe I have upadted both notebooks so they use LocaCluster now like this. Are you looking at the old ones??
client = None
if serial:
    cluster = LocalCluster(**lc_kwargs)
    client = Client(cluster)

client

Also, this is a minor comment, but would you be able to rename your notebooks according to our new suggested naming convention: <region>_<variable>_<metric>_<comparisons>.ipynb (eg, something similar to Global_PSL_NMSE_compare_obs_lens.ipynb or Greenland_SMB_visual_compare_obs.ipynb)

How about these?
month_annual_flow.ipynb -> global_discharge_gauge_comparison.ipynb
ocean_discharge.ipynb -> global_discharge_ocean_comparison.ipynb
For comparison part, it plots observations, but it is not available (because of analysis year), it is not plotted.
what does  mean (is it different from ?)

It looks like the notebooks ran in key_metrics in about 15 minutes. Both notebooks seem to be running smoothly. One additional comment regarding the figures-- in month_annual_flow.ipynb, there are a number of figures that are saved but not directly plotted in the notebook. For the purposes of generating the juptyerbook, I think it'd be best to show the plots directly in the notebook.

15min-> did you run all the components? It seems to take too long. I put a 10 year period (rof_start_data and rof_end_date) in config.yml now, so should be read faster and reading netcdfs is a bottleneck now.
name               Ran?      Elapsed (s)    Percentage
-----------------  ------  -------------  ------------
index              True          9.39778       10.1186
month_annual_flow  True         55.4196        59.6702
ocean_discharge    True         28.0591        30.2112

In the coupled_model example, in Cell 5, there was an error with no files to open. It may be best to leave the key_metrics example, and take out rof from the coupled_model example since we'll need to update that configuration file anyways and create an 'additional metrics' example.

Thanks for clarifying-- the way you are using LocalCluster works!
For the notebook names, would these work? Or if you prefer, maybe you could just take out _compare_obs if observations are only available sometimes?
month_annual_flow.ipynb -> global_discharge_gauge_compare_obs.ipynb
ocean_discharge.ipynb -> global_discharge_ocean_compare_obs.ipynb

RE plotting in line-- I think that's fine if the plots show up when observations are available. Since the default is save_figs=False, I think this is reasonable.
Did you want to take out the coupled_model config file changes, per our discussion on updating the config file to a similar format as the key_metrics config file?

I think the only outstanding comment I'm concerned about is the inclusion of personal directories-- which may need to be pushed to a future issue ticket when we have a more clear location for hosting observational datasets. @mnlevy1981 did you want to take a quick look at this, as well?

I spent some time trying to reduce the number of lines in metrics.py before realizing you never import it.

Thanks for looking at the scripts. Actually it is not imported. I was going to use Cell [19]  where bias, RMSE, and corr are computed in global_discharge_gauge_compare_obs.ipynb.
I put this metrics.py even though they are not used. This computes error metrics, also streamflow metrics (high flow, low flow extreme events etc.), which might be useful (at least use for hydrology, river often).
I can leave this out if confusing now, and add it when (if) need later.

Also, global_discharge_gauge_compare_obs.ipynb ran successfully but

Cell [14] (in 3.2. Annual cycle (at monthly step)) is the last cell that produces output; cells 15 - 21 have the %%time output but no plots or tables

Yes, Cell 15 through 21 are skipped if there are no observations available. For this case, the simulation period year 0001 - 0101, so  it is outside the time period when observation is available. these cells compute error against observations.
in those cells, the second line if obs_available: -> obs_available is False if there is no observation available.

Did some metadata cells get moved around? I see the following sections in order:3. Analysis for 24 large rivers
3.1 Annual flow series
3.2. Annual cycle (at monthly step)
3.3. scatter plots of monthly flow
3.4. scatter plots of annual flow
4. Anaysis for Large 50 rivers
4.1 Summary tables
4.2. scatter plot of annual mean flow

3. Anaysis for all 922 sites
3.1 Compute metris at all the sites (no plots nor tables)
4.2. Spatial metric map
4.3 Boxplots of Error metrics (RMSE, %bias, and correlation coefficient)

Yes, this needs to be updated. But i have been wondering if this is needed?? Also right now each section has a link to the corresponding cell. I don't think the link is not useful here?

(Maybe the reason I could delete metrics.py is because the cells that should call those functions aren't being invoked correctly?)

I haven't had a chance to look at colors.py or utility.py yet, and if it turns out metrics.py is supposed to be called I can go look closer to see if more functions can be replaced with existing numpy calls

utility.py and colors.py are imported (but not all the functions), but metrics.py is not called.
So what I could do is to include the functions used only? These scrips are from the scripts I use for my other notebooks, so if these become useful, could be added later.

2. Did some metadata cells get moved around? I see the following sections in order:
```
3. Analysis for 24 large rivers
3.1 Annual flow series
3.2. Annual cycle (at monthly step)
3.3. scatter plots of monthly flow
3.4. scatter plots of annual flow
4. Anaysis for Large 50 rivers
4.1 Summary tables
4.2. scatter plot of annual mean flow
3. Anaysis for all 922 sites
3.1 Compute metris at all the sites (no plots nor tables)
4.2. Spatial metric map
4.3 Boxplots of Error metrics (RMSE, %bias, and correlation coefficient)
```

Yes, this needs to be updated. But i have been wondering if this is needed?? Also right now each section has a link to the corresponding cell. I don't think the link is not useful here?

I would definitely encourage you to have some markdown cells that describe what plots are being generated. I don't think they need to be organized like a paper with sections and subsections. I hadn't noticed the links from the top of the notebook to each specific section until you mentioned it... let me look at how cupid-build parses those links before we discuss keeping them or removing them.

utility.py and colors.py are imported (but not all the functions), but metrics.py is not called.
So what I could do is to include the functions used only? These scrips are from the scripts I use for my other notebooks, so if these become useful, could be added later.

Yes, please remove functions that are not being used (and then I'm happy to review whatever is left). One reason to provide modules like utility.py and colors.py is to potentially share functions with other notebooks, but if these modules contain code that isn't used anywhere then there is no guarantee that they actually work as advertised.

Did some metadata cells get moved around? I see the following sections in order:

3. Analysis for 24 large rivers
3.1 Annual flow series
3.2. Annual cycle (at monthly step)
3.3. scatter plots of monthly flow
3.4. scatter plots of annual flow
4. Anaysis for Large 50 rivers
4.1 Summary tables
4.2. scatter plot of annual mean flow

Anaysis for all 922 sites
3.1 Compute metris at all the sites (no plots nor tables)
4.2. Spatial metric map
4.3 Boxplots of Error metrics (RMSE, %bias, and correlation coefficient)

Yes, this needs to be updated. But i have been wondering if this is needed?? Also right now each section has a link to the corresponding cell. I don't think the link is not useful here?

I would definitely encourage you to have some markdown cells that describe what plots are being generated. I don't think they need to be organized like a paper with sections and subsections. I hadn't noticed the links from the top of the notebook to each specific section until you mentioned it... let me look at how cupid-build parses those links before we discuss keeping them or removing them.

utility.py and colors.py are imported (but not all the functions), but metrics.py is not called.
So what I could do is to include the functions used only? These scrips are from the scripts I use for my other notebooks, so if these become useful, could be added later.

Yes, please remove functions that are not being used (and then I'm happy to review whatever is left). One reason to provide modules like utility.py and colors.py is to potentially share functions with other notebooks, but if these modules contain code that isn't used anywhere then there is no guarantee that they actually work as advertised.

Ok, I will clean up the scripts (sorry for confusing you.) and the markdown cells.

Hi @mnlevy1981, I think the scripts are clean now, and fixed the errors you pointed out in the notebooks.

Once the comments regarding ancillary data variable naming and personal directories are resolved, we can merge this.

Hi @TeaganKing and @mnlevy1981 , I have updated the notebooks and setup.yaml based on your comments.  thanks!

Great, thanks so much @nmizukami ! This looks good to me. I think merging is blocked due to Mike's change request-- @mnlevy1981 can you please check that this addressed your comments as well and then merge this in?


[Feature] Ability to parse build.cake files to auto create task definitions

I asked the question on Twitter:
https://twitter.com/gep13/status/772878923243876352
And @egamma replied to say that there wasn't currently any extension points to allow extensions to provide additional automatic task definitions.  Instead, that the only point where this was possible was contained within this file:
https://github.com/Microsoft/vscode/blob/master/src/vs/workbench/parts/tasks/common/taskTemplates.ts
So, my question is, would a PR to extend this file to recognise build.cake files be accepted if I were to create one?  Also, I am assuming that I would need to extend the codebase with the ability to parse the build.cake file, but after a quick glance at the code, I couldn't see where that happened, could someone point me in the right direction?  Thanks!

@gep13 Instead of providing a PR to add this to taskTemplates.ts I would suggest to work on a PR to make this an extension point. Simply adding this to the core will not scale.

@dbaeumer agreed. My familiarity with the score base isn't that extensive though. Could you provide some pointers on where that work would be required, and suggestions on how to achieve it? Thanks!

Fully agree with @dbaeumer.
Actually there are several feature similar feature requests for this: #966, #9309.

@egamma is this feature on the roadmap to be implemented by the team, or are you looking for an external contribution?  If an external contribution, would it be possible to get a "starter for 10" in terms of what is required in order to implement the feature?

@gep13 we are just planning the next 2 months, let me get back to you.

@egamma just wanted to follow up with this to see if there was an update. Thanks!

@gep13 the feature didn't make the cut for the September iteration, sorry.

@egamma said...
@gep13 the feature didn't make the cut for the September iteration, sorry.

Thanks for getting back to me!
How close did it get? 😄
Is this feature something that the Core VsCode Team will be looking to implement at "some" point though, or is it something that would be left for a community contribution?

@gep13 it is currently a candidate for October.

Perfect! Let us know if you want an extension to try out the new extensibility points in VsCode. Would be happy to help out.

@egamma how are you?  Just wanted to touch base on this one to see if it is still on the radar for October.  Thanks!

@gep13  As it stands today this will not make it for October.

@dbaeumer said...
As it stands today this will not make it for October.

That's a real shame 😭 I will check back next month.
Thanks for getting back to me though.

@dbaeumer @egamma following the 1.7.1, I thought I would follow up with this again.  Do you know if it is going to make the cut for the next release?  Thanks!

@dbaeumer @egamma just wanted to follow up again.  Thanks!

@gep13: yes and no :-) I started https://github.com/Microsoft/vscode/issues/15179 to discuss how to continue with the task framework, especially around build. The current proposal is that we decouple build from tasks so that extensions can participate in building without providing a task. So for you that would mean that your extensions registers a build provider and can the execute build.cake files without hooking into the task framework. Would that help?

@dbaeumer thanks for getting back to me...
I am not sure if I follow the intention of that issue.  The feature request that I was looking for here was the ability for our Cake extension to "parse" the build.cake file within the currently open folder, and then allow the user to select one of the tasks that are located, by shelling out to the cake.exe.  Is what you are suggesting in #15179 the first step to implementing that, or how does that issue fit into this request.
Thanks!

@gep13: the idea is that you don't need to provide a task to run a build.cake file. You will be able to participate in building (the Ctrl+Shift+B) command and your extension will shell out to cake.exe and do the correct thing.
We also want to move away from one central task file. If you want to provide users access to targets in cake file we are more imagining something like the https://marketplace.visualstudio.com/items?itemName=eg2.vscode-npm-script instead of mangling everything into tasks.
So the idea is also to move the built-in support for gulp into an extension.

@dbaeumer said...
So the idea is also to move the built-in support for gulp into an extension.

Gotcha!  So, if/when gulp becomes an extension, we should be in a position to do this for Cake as well?  Do you have an eta on when that will be happening?
Thanks again!

The work is planned for January next year. Ideas are collected in here  #15179

@dbaeumer perfect, I will follow along there.  Should this issue now be closed then?

I will clean up the issues as soon as the new implementation is ready.

@dbaeumer @egamma I just wanted to follow up with this issue.  Given the recent changes to VsCode, are we now in a position to do what this issue was asking for?  i.e. the ability for an extension author to provide the ability to parse a file (in our case a .cake file) and then allow the execution of the tasks that are located within that file.  If so, could you point me at the documentation for this.
Thanks in advance.

@gep13: unfortunately not. It is on the February iteration plan as a stretch goal. See https://github.com/Microsoft/vscode/issues/20021

@dbaeumer thank you for the update.  If it is a stretch task for February, does that mean if it doesn't appear then, that is will be available in the March update?

@gep13. Yes, that is usually or goal.

@gep13 a fallback option for you is to implement a command that creates a 'task.json' from a cake file.

@dbaeumer @egamma just wanted to follow up with this issue to say that we now have something working:

Which we will hopefully be shipping soon 😄


fix build on CentOS8/aarch64

eventfd() system call is DEPRECATED, use eventfd2() instead!!!
There is no crypto/ec/asm/ecp_sm2z256-armv8.pl  for aarch64

commit a338e3f should use defined(ECP_SM256Z_ASM)  or something like


SimplifyCFG: fix an argument use-after-erase in CheckedCastBrJumpThreading

Need to check if a SILArgument, which was cached earlier is still alive before doing the actual transformation.
rdar://102108656

@swift-ci test

Looks like we need to set each element's parentBlock to nullptr in
void dropAllArguments() { ArgumentList.clear(); }

too.

@swift-ci test


Create o365sen.txt

@Applekr This domain does not belong to a university, nor to a college. I have to close the request without merging. Please ensure that you are familiar with this repository rules located at the bottom of this page before submitting a new request.
Also, WHOIS information does not indicate that this school is eligible for adding to this repository:
Domain Name : o365sen.net
Registry Domain ID:<PHONE_NUMBER>_DOMAIN_NET-VRSN
Registrar WHOIS Server: whois.yesnic.com
Registrar URL: http://yesnic.com
Updated Date: 2020-01-22T08:54:26Z
Creation Date: 2015-02-04T05:58:47Z
Registrar Registration Expiration Date: 2021-02-04T05:58:47Z
Registrar: Whois Networks Co., Ltd.
Registrar IANA ID: 100
Registrar Abuse Contact Email<EMAIL_ADDRESS>Registrar Abuse Contact Phone: +82.15884259
Reseller:
Domain Status: clientTransferProhibited https://icann.org/epp#clientTransferProhibited
Registry Registrant ID:
Registrant Name: pobistnc
Registrant Organization: pobistnc
Registrant Street: 22 Hakdong-ro 19-gil Gangnam-gu
Registrant City: Seoul
Registrant State/Province:
Registrant Postal Code: 06045
Registrant Country: KR
Registrant Phone: +82.7046167042
Registrant Phone Ext:
Registrant Fax:
Registrant Fax Ext:
Registrant Email<EMAIL_ADDRESS>Registry Admin ID:
Admin Name: pobistnc
Admin Organization: pobistnc
Admin Street: 22 Hakdong-ro 19-gil Gangnam-gu
Admin City: Seoul
Admin State/Province:
Admin Postal Code: 06045
Admin Country: KR
Admin Phone: +82.7046167042
Admin Phone Ext:
Admin Fax:
Admin Fax Ext:
Admin Email<EMAIL_ADDRESS>Registry Tech ID:
Tech Name: pobistnc
Tech Organization: pobistnc
Tech Street: 22 Hakdong-ro 19-gil Gangnam-gu
Tech City: Seoul
Tech State/Province:
Tech Postal Code: 06045
Tech Country: KR
Tech Phone: +82.7046167042
Tech Phone Ext:
Tech Fax:
Tech Fax Ext:
Tech Email<EMAIL_ADDRESS>Name Server: ns1.whoisdomain.kr
Name Server: ns2.whoisdomain.kr
Name Server: ns3.whoisdomain.kr
Name Server: ns4.whoisdomain.kr
DNSSEC: unsigned


Support static Docker image as base for local universe.

For audit purposes, it's easier to have a single base image to audit.  This adds the following potential workflows:
The default build process generates new certificates and a new universe-base image every time it's run:
make certs              # Generates certificates
make base               # Uses certificates to generate universe-base
make local-universe     # Uses universe-base and adds content to generate local-universe

This means that every time you change certificates, you have a different 'universe-base' container, which is problematic for audit purposes.
The new process follows this workflow:
make static            # One-time run to generate universe-static
make static-base       # Used to add certificates to universe-static (and generate universe-base)
make local-universe    # Uses universe-base and adds content to generate local universe

Now, only the universe-static image has to be audited.
Alternately, a static image at justinrlee/universe-static:0.1 can be used, as follows:
# Download justinrlee/universe-static:0.1
docker pull justinrlee/universe-static:0.1

# Re-tag as universe-static:latest
docker tag justinrlee/universe-static:0.1 universe-static:latest

# Add certificates (generate universe-base from universe-static)
make static-base

# Add content (generate local-universe from universe-base)
make local-universe    # Uses universe-base and adds content to generate local universe

I'd also like to have a a mesophere/universe-static:0.1 Docker image, but I'm not sure how to get that done.  Can be retagged from justinrlee/universe-static:0.1 (I don't believe I have permissions to push to /mesophere on Docker hub).
If I can get this done, I'd like to add a third new make target: make online-static, which is literally just a wrapper around 'docker pull mesosphere/universe-static:0.1 && docker tag mesosphere/universe-static:0.1 universe-static:latest`

JIRA: https://jira.mesosphere.com/browse/DCOS-16014

@justinrlee check your Slack for how to get the Docker Hub credentials

@justinrlee very cool!

I'd also like to have a a mesophere/universe-static:0.1 Docker image, but I'm not sure how to get that done. Can be retagged from justinrlee/universe-static:0.1 (I don't believe I have permissions to push to /mesophere on Docker hub).

Can we make sure that this repo contains the Dockerfile for generating this static image. Let's also make sure that we have CI that can generate that image and uploads it to Docker Hub. In other words, this image should be reproducible through our CI. If we are not already doing that please work with @cruhland to get that done.

If I can get this done, I'd like to add a third new make target: make online-static, which is literally just a wrapper around docker pull mesosphere/universe-static:0.1 && docker tag mesosphere/universe-static:0.1 universe-static:latest

Why do you need docker tag ... universe-static:latest? Our dependencies should be to static and immutable images.

@jsancio

Can we make sure that this repo contains the Dockerfile for generating this static image. Let's also make sure that we have CI that can generate that image and uploads it to Docker Hub. In other words, this image should be reproducible through our CI. If we are not already doing that please work with @cruhland to get that done.

Yeah, I can't figure out how to modify the Mesosphere docker hub repos.  Not sure if it's a permissions thing or something else.  We also don't actually want CI generating this - it should be a static image (e.g., should not be built and re-built).
Why do you need docker tag ... universe-static:latest? Our dependencies should be to static and immutable images.
Eh, the Dockerfile.static.base has a from universe-static in it, to use whichever is latest.  I thought if it had a non-latest flag, it wouldn't work?

@jsancio do we want the CI job to run on every commit, or just be manually triggered? As @justinrlee said, we don't need to rebuild and re-upload a particular tag if it's already present on Docker Hub.

@jsancio do we want the CI job to run on every commit, or just be manually triggered? As @justinrlee said, we don't need to rebuild and re-upload a particular tag if it's already present on Docker Hub.

@justinrlee and @cruhland I think we need to manually trigger it. What is the process for changing the static images?

Change the docker file
Trigger a CI build with the new version. This build:

Builds the new static image
Uploads it to Docker Hub

Change the Universe repo to reference the new images

Is that it? What else?

@jsancio that looks correct to me. @justinrlee could you address my comments so we can get this merged? I'll help you create the manual CI build afterwards.


Adds jbang template to create reproducers

With this you just do jbang init -t reproducer@hibernate/hibernate-reactive mytest
and you will have a mytest.java.
Advantage is you can create any number of reproducers and you get your own file rather than
editing the file in jbang's internal cache.
Other fixes:

move #! at top of file as otherwise it breaks ability to run it directly.
add simple sl4j to get useful output rather than warnings. you can also use sl4fj-nop if you just want it quite.

I guess jbang doesn't create the file wihtout the init and the template

Merged, thanks

Doesn't this give me the same result?

no - it downloads the testcase into jbang cache and what you are editing is the file directly in jbang cache. meaning no guarantees when that file will be nuked/moved/overwritten.
I probably should try and prevent that edit operation in jbang.

Couldn't you add an option instead?

jbang edit --live --open=idea  testcase@hibernate/hibernate-reactive Issue123.java

It's just a minor thing though

What do you mean default name ?
You just list a name - which for a reproducer would be something relevant for the issue.
Or you prefer every producer you get has same name? :)

Or you prefer every producer you get has same name? :)

Do I really care? Most of the time I want to try something and then throw it away.
Or even if different users send me two reproducer with the same name, I hope that the code will be at least different.
The default name is only for when the user doesn't set it: It's going to be IssueTest.java if not set or HR123Test.java if set.
Most of the users will only send a reproducer and then disappear.

Maybe I'm wrong but with a default name you could also run a template

By the way, I haven't used JBang that much so it's very possible that what I'm saying doesn't make sense.
For some reason, I just find it weird that I have to use a template when the only things that changes is the name of the class. While, if I use an alias I can just run or open the file without any additional steps.
But if that's the way it is, I will get used to it :-)


fix: add options parameter to AppContainer

Pull Request Guidelines
Our guidelines for submitting a pull request.
Before submitting a Pull Request, please make sure you have verified the following:

[x] The commit message follows our guidelines:

A good commit message should be two things: meaningful and concise. It should not contain every single detail, describing each changed line—we can see all the changes in Git—but, at the same time, it should say enough to avoid ambiguity.
We use Microverse's commit message convention
The convention stablish that a commit message has to be in the present tense, imperative and lowercase.
Example: fix typo in README.md

[x] Tests for the changes have been added (for bug fixes / features)
[x] Docs have been added / updated (for bug fixes / features)

PR Type
What kind of change does this PR introduce?

[ ] Bugfix
[x] Feature
[ ] Code style update (formatting, local variables)
[ ] Refactoring (no functional changes, no api changes)
[ ] Build related changes
[ ] CI related changes
[ ] Other... Please describe:

What is the current behavior?
Please describe the current behavior that you are modifying, or link to a relevant issue.
Issue Number: N/A
What is the new behavior?
Allows set options to Inversify's container or override default options defined by ExpressoTS
To do that today:
const appContainer = new AppContainer();

const container = appContainer.create([
  // Add your modules here
  UserModule
]);
/**
 * Update options AFTER call `appContainer.create`,
 * do it before throw `TypeError: Cannot read properties of undefined (reading 'options')`
 * because Container only is instanced after `appContainer.create` is called
 */
appContainer.Container.options.skipBaseClassChecks = true;

More intuitive solution:
const appContainer = new AppContainer({ skipBaseClassChecks: true })

// ... code

Does this PR introduce a breaking change?

[ ] Yes
[x] No

If this PR contains a breaking change, please describe the impact and migration path for existing applications below.
Other information
Any other information that is important to this PR.

interfaces.ContainerOptions offer more options than we want to expose to the user at the moment. We will do so based on demand.
For now, what I did constrain the options you can pass in the constructor:
interface ContainerOptions {
  /**
   * The default scope for bindings in the container.
   * It can be set to Request (default), Singleton, or Transient.
   */
  defaultScope?: interfaces.BindingScope;

  /**
   * Allows skipping of base class checks when working with derived classes.
   */
  skipBaseClassChecks?: boolean;
}

And by default the container will have the settings:

autoBindInjectable: true,
defaultScope: Request

Now, the user can pass in the constructor those to options:

defaultScope and SkiLibBaseCheck

I removed the default scope from the create method

Doc is up to date: https://doc.expresso-ts.com/pt/docs/overview/app-container


complete rewrite + nextjs

:tada: This PR is included in version 1.3.0 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


[bug (/docs?)] Validators are being run earlier than FORCE_ENV_FOR_DYNACONF can switch to testing env, thus, failing

Describe the bug
Currently, it seems not possible to use pytest fixture with FORCE_ENV_FOR_DYNACONF if the validators exist, due to validators run earlier and might fail.
Note, that in the app's codebase (as well as tests/conftest.py) I import config as:
from mypkg.config import settings
and not the
from dynaconf import settings
Because it seems like validators will not be run at all if are imported the other way.
(What is the downside of such import and might that be a source of the subject's problem?)
Below is the complete example of the environment when the subject happens.
To Reproduce
Steps to reproduce the behavior:

Having the following folder structure

 Project structure

# /mypkg/
# ...__init__.py...
# ...config.py...
# ...<other stuff>
# /tests/
# ...__init__.py...
# ...conftest.py/...
# ...test_010.py/...
# settings.toml
# .env

Having the following config files:

 Config files
/.env
MYPKG_ENV=production

<other non-python-app specific vars>

and
/settings.toml
[default]

[testing]
TMP_ROOT_DIR="/tmp/mypkg/tests"

[production]
# For production we'll just use [default] and some env vars from Dockerfile

Having the following app code:

 Code
/mypkg/config.py
from dynaconf import Dynaconf, Validator

validators = [
    # Ensure some parameters exists (are required)
    Validator(
        "TMP_ROOT_DIR",
        must_exist=True,
    ),
]

settings = Dynaconf(
    validators=validators,
    envvar_prefix=False,
    settings_files=["settings.toml", ".secrets.toml"],
    environments=True,
    load_dotenv=True,
    env_switcher="MYPKG_ENV",
)

conftest.py:
import pytest
from mypkg.config import settings

@pytest.fixture(scope="session", autouse=True)
def set_test_settings():
    """Forces usage of [testing] env from `settings.toml` for tests"""
    settings.configure(FORCE_ENV_FOR_DYNACONF="testing")

Current behavior
The ValidationError is raised before settings.configure() can be executed.
Expected behavior
To be able to use settings.configure() to actually switch the env and then run the validators
Environment (please complete the following information):

OS: Ubuntu 20.04
Dynaconf Version 3.1.4
Frameworks in use: pytest

Additional questions

Is it okay to import settings directly from config.py and not from dynaconf module? Why the validators do not work when importing directly? Seems like a separate bug issue.
Are [default] settings implicitly inheritable? (Meaning everything that is not in [production] but in [default] will be available in [production] as well? Is that documented somewhere? Is there a switch to turn it on/off?)

Yep, I've been banging my head against this one for days too.
And it seems that running settings.validators.validate() after the settings.configure() does nothing too, even though settings.current_env shows the right "FORCED" env.

I've created a quick repo for this with a pytest action recreating the issue, @rochacbruno .
Test result here

I also noticed this text in the docs:

By default, if the data of an instance is updated with update, set or load_file methods, no validation will be triggered.

You can override this globally with the option validate_on_update or set this on a per-call basis.

If I set either validate_on_update=True on instantiation, or as:
@pytest.fixture(scope='session', autouse=True)
def set_test_settings():
    current_directory = os.path.dirname(os.path.realpath(__file__))
    settings.configure(FORCE_ENV_FOR_DYNACONF='testing',
                       VALIDATE_ON_UPDATE_FOR_DYNACONF=True
                       )
    settings.load_file(path=['proxmox_grapple_tests.yml'], silent=True, validate=True)

I get:
E   RecursionError: maximum recursion depth exceeded
!!! Recursion detected (same locals & position)

Test result here.

I believe this might not be the right use for the configure method, because its recommended to be used for django context (by the docs, dont know much about this part of the implementation).
Anyway, I'll have a better look into this. Thanks for the providing rich context on the problem!

Hi @pedro-psb, thanks for looking into it.

I believe this might not be the right use for the configure method, because its recommended to be used for django context (by the docs, dont know much about this part of the implementation).

The docs seem to indicate configure can be used with pytest; but I think it assumes you have one settings file, for both production and testing. I'm trying to make it load a testing-only settings file (as well as change running env as per this whole PR).
I'm trying to write a functional test to make sure my app catches validation errors and "does the right stuff", and right now, even with a intentionally-broken settings file, the tests are saying "all is well."

I took a look and experimented with this:
Observation
1. deprecated global settings
This global settings is deprecated since 3.0.0 (just so you know)
from dynaconf import settings

2. importing settings from other module
That is not a problem if in the config module settings is instantiated with Dynaconf (not global settings), as it is in your examples:
from config import settings # but config uses dynaconf.Dynaconf, not dynaconf.settings

3. testing good pratices
Not a good practice to import a module with side-effects (instatiating dynaconf, in this case) on the top-level of a test or conftest, because:

you loose control over how the instantiation should be setup.
you get poor test isolation as you have a mutable object in the wild (outside pytest fixtures world)

4. documentation is out of date
As for the observation about global settings deprecation, we can state that this section of the docs is really outdated.
Alternatives
I don't think there is any reason to try fixing this specific approach, as those are based on outdated docs. But there are some alternatives.
a) Explicit fixture
import pytest

@pytest.fixture
def settings(monkeypatch):
    monkeypatch.setenv("ENV_FOR_DYNACONF", "testing")
    # load module here (in a controlled test environment)
    from config import settings as _settings
    return _settings

def test_running_on_testing_environment(settings):
    assert settings.current_env == "testing"
    assert settings.ENV_FOR_DYNACONF == "testing"

b) Implicit fixture
import os
from unittest import mock

import pytest

@pytest.fixture(scope="session", autouse=True)
def set_settings():
    # cant use monkeypatch with session-scoped fixtures
    with mock.patch.dict(os.environ, {"ENV_FOR_DYNACONF": "testing"}, clear=True):
        yield

def test_running_on_testing_environment():
    # load here instead of on the top-level
    # so we have control over the environ
    from config import settings

    assert settings.current_env == "testing"
    assert settings.ENV_FOR_DYNACONF == "testing"

Outcomes
If those alternatives work for you, I think we can change this issue to a docs issue and replace the recommended approach to setup pytest without a framework.
Please, let me know what you think!

I'll preface this by saying I'm not sure if you're replying at me, or at @Larleyt, but will assume me.

I took a look and experimented with this:
Observation
1. deprecated global settings This global settings is deprecated since 3.0.0 (just so you know)

Understood, I'm not using it that way.

2. importing settings from other module That is not a problem if in the config module settings is instantiated with Dynaconf (not global settings), as it is in your examples:

Sorry, who's examples?  I'm not seeing it used that way in my examples...

from config import settings # but config uses dynaconf.Dynaconf, not dynaconf.settings

Sorry, your code comment here contradicts your first statement above?

3. testing good pratices Not a good practice to import a module with side-effects (instatiating dynaconf, in this case) on the top-level of a test or conftest, because:

you loose control over how the instantiation should be setup.
you get poor test isolation as you have a mutable object in the wild (outside pytest fixtures world)

Or, is it a design flaw that nothing should really happen at instantiation until the code is asked to do something?

In my experimentation, it seems that Dynaconf lazily executes the validation (even when configuring right at instantiation), seemingly waiting until the settings instance is first accessed.  This goes against this part of the doco.

4. documentation is out of date As for the observation about global settings deprecation, we can state that this section of the docs is really outdated.

:+1:

Alternatives
I don't think there is any reason to try fixing this specific approach, as those are based on outdated docs. But there are some alternatives.
a) Explicit fixture

An interesting approach, and similar to how I've fixed it "for now".  I merely don't run my fixture setup code at all, and just use this in my tox.ini:
set_env =
    FORCE_ENV_FOR_DYNACONF = testing
    SETTINGS_FILE_FOR_DYNACONF = tests/proxmox_grapple_tests.yml

b) Implicit fixture

Yeah, very convoluted though, no?

If those alternatives work for you, I think we can change this issue to a docs issue and replace the recommended approach to setup pytest without a framework.

Not sure what you mean about "framework".
Overall, I still don't feel like any of this is clean.

Thanks for the feedback.
I was referring to both examples, as they are similar in general. But I was hoping that you would answer, as this issue is pretty old.

Sorry, who's examples? I'm not seeing it used that way in my examples...

Sorry, this section came out pretty confusing. I meant that your example is ok.

About the validation behavior, the docs is wrong again, we need to fix it. The validation is triggered upon access.
And I believe I found the reason why validation was being fired early. The importing from config import settings did nothing special by itself, but Pytest does some inspecting on the test files on the test collection process, and that was accessing the imported (lazy) settings and firing the validation. So again, IMO the solution for that is proper isolating the instantiation/import.

b) Implicit fixture

Yeah, I prefer a), but who knows.

Not sure what you mean about "framework".

Sorry, "webapp" frameworks: Django, Flask or FastApi. There might be specific test setups for those.

Overall, I still don't feel like any of this is clean.

I like option a), but it would be nice to hear how do you think could be a clean way. Maybe we can investigate if we can support that.

Sorry, this section came out pretty confusing. I meant that your example is ok.

All good!

About the validation behavior, the docs is wrong again, we need to fix it. The validation is triggered upon access.
And I believe I found the reason why validation was being fired early. The importing from config import settings did nothing special by itself, but Pytest does some inspecting on the test files on the test collection process, and that was accessing the imported (lazy) settings and firing the validation. So again, IMO the solution for that is proper isolating the instantiation/import.

Or... option C... why not a new kwarg to settings.configure to disable early/lazy validation?  And then the user has to explicitly call settings.validators.validate()?  A bit like how the doco states you can configure validators later on?
Or would the kwarg to settings.configure already be too late?

Yeah, I prefer a), but who knows.

Me too, thus far, in lieu of anything else.  I'm just with oldschool Python that (used to? still does?) say good practice isn't to import anywhere but at the top of your code.

I like option a), but it would be nice to hear how do you think could be a clean way. Maybe we can investigate if we can support that.

Although the lazy validation triggered at first access is "cute", I also think it's a bit of magic/voodoo that introduces more complexity and confusion.  I personally wouldn't expect a library to do almost anything in import, or on first access.
I feel validations should always be explicitly requested.  User configures the settings, validators etc, and then needs to "click start" so to speak.  Having code raise an exception before that time also makes it weirder/harder to catch.

settings done in a config.py
Main code imports settings
Main code does other stuff, setup logging, click stuff, etc
Then finally, a simple:

try:
    settings.validators.validate()
except ValidationError as e:
    click.echo(e, err=True)
    sys.exit(1)

A kwarg could also ease migration by defaulting to existing behaviour and not breaking code already out there.

Or... option C... why not a new kwarg to settings.configure to disable early/lazy validation? And then the user has to explicitly call settings.validators.validate()? A bit like how the doco states you can configure validators later on?
Or would the kwarg to settings.configure already be too late?

I guess not, your idea should be doable. But you can configure validations later with this approach and make an explicit call to validate. I'm not sure if I follow your motivation.

About your idea of making things more explicit, I'm in favor of that. I'm a relatively new maintainer of this project, but from what I've experienced and understood about the codebase so far, this idea of triggering things upon access is not just some personal flavor, it probably has to do with the integration with web frameworks, specially django. Django doesn't provide a clean way to customize their settings, and due to its long-living-webserver nature, sometimes it's pretty challenging to support integration with them.
But yeah, your idea sounds good and reasonable, thanks for sharing it!

Or... option C... why not a new kwarg to settings.configure to disable early/lazy validation? And then the user has to explicitly call settings.validators.validate()? A bit like how the doco states you can configure validators later on?
Or would the kwarg to settings.configure already be too late?

I guess not, your idea should be doable. But you can configure validations later with this approach and make an explicit call to validate. I'm not sure if I follow your motivation.

My motivation is being explicit I guess.  I kinda personally feel the lazy validation method linked above is more of this "magic" I've mentioned.  I think configuring the validators using the config.py settings = Dynaconf() model keeps most of Dynaconf in one place, is clean, and then you can (optionally) validate when you want to, not when magically first accessed.
It of course takes away any magic for functional/integration testing.

About your idea of making things more explicit, I'm in favor of that. I'm a relatively new maintainer of this project, but from what I've experienced and understood about the codebase so far, this idea of triggering things upon access is not just some personal flavor, it probably has to do with the integration with web frameworks, specially django. Django doesn't provide a clean way to customize their settings, and due to its long-living-webserver nature, sometimes it's pretty challenging to support integration with them.

Copy that, understood.
If this becomes an impossible to fix issue we must mark as a known-caveat on the django docs and close the issue

#1157 will change how multiple environments work and must address this issue


modules/gtk: add more ports to gtk label

So it can be used to simulate led 7 segments.
Signed-off-by: Bruno Dilly<EMAIL_ADDRESS>

new version on #790


Install Binn folder (ssbdiagnose and other \Tools\Binn)

I have installed SSMS 18.5
But ssbdiagose and other utilities not installed from downloaded installation file (18.5)
I've copied C:\Program Files (x86)\Microsoft SQL Server\140\Tools\Binn from other PC
Now error not found assembly
Microsoft.SqlServer.Smo, Version=<IP_ADDRESS>
How to solve this?

@IFetisov -- thank you for your question.  Are you following a particular docs.microsoft.com article?  Meanwhile, please try these resources:

DBA Stack Exchange
Stack Overflow

You are welcome to report this issue here:

SQL Server Feedback


Nd deconv GPU support

Can we add some gpu test for 1D and 3D?

I plan on adding tests once the cpu implementation is complete.

Can we add some gpu test for 1D and 3D?

I plan on adding tests once the cpu implementation is complete.

So there is another PR for the CPU implementation?

Can we add some gpu test for 1D and 3D?

I plan on adding tests once the cpu implementation is complete.

So there is another PR for the CPU implementation?

Yes, I plan to open another PR with those changes including cpu/gpu tests.


[Slider] Fix mouse enter edge case for Firefox

As suggested by @oliviertassinari in https://github.com/mui-org/material-ui/issues/16983#issuecomment-520814056

[x] I have followed (at least) the PR section of the contributing guide.

Closes #16983

It's a great first contribution to Material-UI. 🎉 Thanks for working on it :)


(WIP) Add DirectML and Phi-3 support to Genny example

Fixes #569

@microsoft-github-policy-service agree

@microsoft-github-policy-service agree

Hi! Thanks for the contribution. The CI pipeline was broken yesterday and fixed now. Feel free to re-run the failed jobs.


Add "preferred" identifiers to bulk download

Hello - we've started using your KGX bulk download for the NodeNormalizer, and it's been great - thanks for making that!
However, we still have to also query your API because there doesn't seem to be any indication of the "preferred" identifier for each group of equivalent nodes in the bulk download.
For example, the "preferred" identifier for the concept 'water' seems to be PUBCHEM.COMPOUND:962, which is reported under [input_curie] --> "id" --> "identifier" in the NodeNormalizer RestAPI /get_normalized_nodes response:
curl -X 'GET' \
  'https://nodenormalization-sri.renci.org/1.3/get_normalized_nodes?curie=MESH%3AD014867&conflate=true' \
  -H 'accept: application/json'

{
  "MESH:D014867": {
    "id": {
      "identifier": "PUBCHEM.COMPOUND:962",
      "label": "Water"
    },
    "equivalent_identifiers": [
      {
        "identifier": "PUBCHEM.COMPOUND:962",
        "label": "Water"
      },
      {
        "identifier": "CHEMBL.COMPOUND:CHEMBL1098659",
        "label": "WATER"
      },
      {
        "identifier": "UNII:059QF0KO0R",
        "label": "WATER"
      },
      ...

While in the bulk JSON lines KGX nodes file, I see equivalent_identifiers, but no indication of the "preferred" identifier for each cluster:
{"id": "UMLS:C0448837", "name": "Skin structure of female perineum", "category": "biolink:AnatomicalEntity", "equivalent_identifiers": ["UMLS:C0448837"]}
{"id": "UMLS:C0447166", "name": "Lower jugular lymph node", "category": "biolink:AnatomicalEntity", "equivalent_identifiers": ["UMLS:C0447166", "NCIT:C132512"]}
{"id": "NCIT:C132512", "name": "Lower Jugular Lymph Node Group (Level IV)", "category": "biolink:AnatomicalEntity", "equivalent_identifiers": ["UMLS:C0447166", "NCIT:C132512"]}
{"id": "UMLS:C4020414", "name": "subcentral operculum", "category": "biolink:AnatomicalEntity", "equivalent_identifiers": ["UMLS:C4020414"]}
{"id": "UMLS:C0832304", "name": "Bone of head of phalanx of right middle finger", "category": "biolink:AnatomicalEntity", "equivalent_identifiers": ["UMLS:C0832304"]}
{"id": "UBERON:4300154", "name": "procurrent spur", "category": "biolink:AnatomicalEntity", "equivalent_identifiers": ["UBERON:4300154"]}
{"id": "UMLS:C0821524", "name": "Trabecular bone of left pedicle of twelfth thoracic vertebra", "category": "biolink:AnatomicalEntity", "equivalent_identifiers": ["UMLS:C0821524"]}
{"id": "UMLS:C0924689", "name": "Trabecular bone of proximal phalanx of left third toe", "category": "biolink:AnatomicalEntity", "equivalent_identifiers": ["UMLS:C0924689"]}
{"id": "UMLS:C1283383", "name": "Spinothalamic tract of medulla", "category": "biolink:AnatomicalEntity", "equivalent_identifiers": ["UMLS:C1283383"]}
{"id": "UMLS:C0224428", "name": "Entire tensor fascia lata", "category": "biolink:AnatomicalEntity", "equivalent_identifiers": ["UMLS:C0224428"]}

It would really help us out if you could add this. Thanks!

@cbizon Would it be okay if I added a preferred_id node property to store the preferred id for any clique? Or is there an existing KGX property that would be better suited for this?

Isn't "id" the preferred id?

@amykglen -- I think Chris is correct, the id is the preferred ID for each clique. Does that meet your needs?

yes, it does. thank you!


Make GCMRegistrar.checkManifest() aware of Jelly Bean changes

If an app is targeted to Jelly Bean:

- The broadcast receiver does not need the category
- The C2D_MESSAGE is not required

GCMRegistrar.checkManifest() should take the version in account and disable
these 2 checks when running on target 16.

Original issue reported on code.google.com by<EMAIL_ADDRESS>on 7 Aug 2012 at 7:08

Please migrate to the new Google Play Services InstanceID API.


Remove GrB_vector

Codecov Report

Merging #56 (68b0490) into master (80d15b0) will decrease coverage by 3.31%.
The diff coverage is 60.55%.

@@            Coverage Diff             @@
##           master      #56      +/-   ##
==========================================
- Coverage   29.18%   25.86%   -3.32%
==========================================
  Files          44       44
  Lines        4523     4454      -69
==========================================
- Hits         1320     1152     -168
- Misses       3203     3302      +99

Impacted Files
Coverage Δ

src/SuiteSparseGraphBLAS.jl
88.88% <ø> (ø)

src/lib/LibGraphBLAS.jl
14.99% <ø> (-5.82%)
:arrow_down:

src/operations/argminmax.jl
0.00% <0.00%> (ø)

src/operations/concat.jl
0.00% <0.00%> (ø)

src/operations/kronecker.jl
60.86% <ø> (ø)

src/operations/resize.jl
0.00% <0.00%> (ø)

src/operations/transpose.jl
83.87% <ø> (+16.30%)
:arrow_up:

src/matrix.jl
50.28% <33.33%> (-0.81%)
:arrow_down:

src/print.jl
16.94% <38.46%> (-40.95%)
:arrow_down:

src/options.jl
18.75% <50.00%> (ø)

... and 18 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 80d15b0...68b0490. Read the comment docs.


Complete removal of jQuery. Resolves #538

Found a bug: (1) go to Streetmix (2) open an external URL in the same tab (e.g. with a browser bookmark) (3) click "back." Streetmix won't load.

Can't seem to duplicate the problem I was experiencing -- I'll merge this in but keep an eye on whether it crops up again.


Update items.json

Changed order of item 140 and 141 to follow numeral chronology (OCD kicked-in, sorry).
Also, added the Key item entries for all three Medal of Honor bonuses you get after defeating all Moldugas, Hinox's and Talus' in the game.
There is still one other key item I know I'm missing, that's Obj_ProofKorok.
I still don't know what its name is in the game, I will add it once I know it. :P

Top man, thanks mate - OCD is fine with me!


Update Composer dependencies (2021-07-17-00-05)

Visual regression test passed!
View the visual regression test report


Add StandardJS (closes #323)

This PR adds StandardJS (#323) with the configuration for this project.
I've adapted the code under the test folder to that code style too. The rest of the code should be done incrementally because it is composed of a lot of different parts.

@spresnal In that case I kept it that way because I'm waiting to having more parts refactored (https://github.com/ArkEcosystem/ark-desktop/issues/298) before changing to a more recent Node.js version (https://github.com/ArkEcosystem/ark-desktop/issues/299)

Merged manually! +5 👍🏻


[ auto submit ] auto submit bot does not submit successfully

For PR https://github.com/flutter/flutter/pull/109172 , I waited for 4 hours but it didn't get submitted.

I am expecting https://github.com/flutter/cocoon/pull/2035 solves the issue.

Closing as obsolete.


Correct Fastly

Fastly subdomains can be taken over. Works as of 5 minutes ago.

Also adding Bitbucket & Ghost

Thank you very much, @haccer!


Add fuzzy search to Table Of Contents

Add fuzzy search to Table Of Contents

@zammer thanks for the PR, I love the feature! But please look at the comments in the source code. (Maybe you'd like to wait for more input from @sapegin before you start fixing it).

@zammer please also fix the reported ESLint errors!

And thanks for the PR @zammer!

No worries.
Leave it with me.

Shouldn't
const regex = new RegExp(value.toLowerCase().toString(), 'g');
be
const regex = new RegExp(value.toString().toLowerCase(), 'g');
instead? A value that isn't a string already will throw if toLowerCase is called on it...

@sapegin and @mik01aj refactored as requested.

LGTM :shipit:

I’ll tweak it a bit later. Congrats with the first pull request!
:confetti_ball: :confetti_ball: :confetti_ball: :confetti_ball: :confetti_ball:

Thankyou


fix(cmd): kong vault get doesn't work in dbless mode

Summary
The cli kong vault get <reference> doesn't work in DBless mode if <reference> uses vaults entity. It doesn't affect the normal use of vault in kong instance though.
The reason is in DBless mode the vaults entity is stored in LMDB which is implemented by a Nginx C module. However Everytime resty cli (which is relied on by kong cli) runs it creates a temporary nginx.conf which doesn't contain the lmdb-related directives.
This PR is fixing this by starting another resty call with the lmdb-related directives of the running Kong inserted into the temporay nginx.conf that resty will create via the --main-conf option, so that it can access the correct initialized LMDB.
Note we only try this after detecting this specific error no LMDB environment defined in order to avoid infinite loop. And because resty creates a temmporary nginx instance so we need to convert the relative paths in the nginx.conf to the absolute path under kong instance prefix.
FTI-4937

We, @kikito, @jschmid1, and @bungle, decided to move this to 3.4. This is a rather generic problem, and we would like to pursue different approaches before committing to this. Approaches that might work in more generic fashion are:

patching resty (to include lmdb stuff, and perhaps lua trusted certs)
changing bin/kong to run in shell and then start the resty with required attributes
there is a related fix in ee: https://github.com/Kong/kong-ee/pull/5215 that tries to fix lua trusted certs problem with luasocket that reads system certs (which fixes only that part of this problem)

Agree. The current fix is a bit hacky. Patching resty and bin/kong would be a more generic way.

@bungle The tricky part of this problem is that we don't really know what directives to inject before getting the kong conf by calling conf_load. So we need to run a process first, resty or whatever, to get this information, and then compile this information to obtain the corresponding nginx conf. It feels like the chicken-and-egg question. This limitation seems always exist as long as our cmd implementation still relies on resty. Actually, kong start command is implemented with a similar idea. The different part is kong start calls nginx eventually, while kong vault calls resty again (self-call).

Here is a draft of generic fix:
https://github.com/Kong/kong/pull/10785

Updated this PR to make it more generic. The basic idea keeps the same, but the main processing logic is moved from kong/cmd/vault.lua forward to kong/cmd/init.lua so that other commands can reuse this part of logic and if necessary we add other directives conveniently.
TODO: add more tests.

Because of the reason mentioned above, I don't think statically adding directives into resty_cli will work. The config can be specified by env or custom config file or inherit from the default configuration. And some preparation like .ca_combined may also be required. So it's still most convenient and maintainable to load and compile the conf in Lua land. @bungle What do you think?


fix bugs on android armv7, because pw.pw_passwd.is_null()

crash on my Android phone because pw.pw_passwd == NULL

How are you testing these changes - something like Termux? I'd like to see if I can reproduce this on my end.
Thanks!

@rtzoeller  use adb which in Android SDK like below:
$ adb push <path-to-exec> /data/local/tmp
$ adb shell /data/local/tmp/<exec-file>

Rust code:
use nix::unistd::{self, User};

fn main() {
    let user = User::from_uid(unistd::getuid())
        .expect("getpwuid_r()")
        .expect("option");
    println!("{:?}", user);
}

Crash also happened on aarch64 android platform.

Thanks! Confirmed this reproduces for me.
Can you add an entry to the CHANGELOG.md file under the Fixed section, squashed into the current commit?

@ClarkGuan can you squash the current two commits into a single commit?

@rtzoeller OK.


Resolve syncing conflicts from live-sync-work to live

Pull request opened by Docs to resolve syncing conflicts from live-sync-work to live.

Docs Build status updates of commit ec7274d:
:x: Validation status: errors
Please follow instructions here which may help to resolve issue.

File
Status
Preview URL
Details

:x:Error

Details

[Error] Cannot sync git repo to specified state. Please make sure the pull request is mergeable. Details: Git merge results in conflict, repository remote url: https://github.com/OPS-E2E-PPE/azure-docs-sdk-dotnet, working directory: W:\dv4x-s, target branch: HEAD, target branch commit: 04c085e79d324602feccc0617fba74219c80ff12, source commit: ec7274d59322b7c6503ec102bf8b9670887a80bb.

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.

Docs Build status updates of commit ec7274d:
:x: Validation status: errors
Please follow instructions here which may help to resolve issue.

File
Status
Preview URL
Details

:x:Error

Details

[Error] Cannot merge commit ec7274d59322b7c6503ec102bf8b9670887a80bb in branch live-sync-work of repository https://github.com/OPS-E2E-PPE/azure-docs-sdk-dotnet into branch live (commit 04c085e79d324602feccc0617fba74219c80ff12). Please follow this documentation: https://help.github.com/articles/resolving-a-merge-conflict-using-the-command-line/ to use git.exe to resolve you content conflicts locally and then push to remote.

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.

Docs Build status updates of commit 5df32a1:
:x: Validation status: errors
Please follow instructions here which may help to resolve issue.

File
Status
Preview URL
Details

:x:Error

Details

[Error] Run script failed with no detailed information. Please retry building your content again. If the issue still happens, open a ticket in http://SiteHelp and include the error report for our team to troubleshoot

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.

Docs Build status updates of commit 5df32a1:
:x: Validation status: errors
Please follow instructions here which may help to resolve issue.

File
Status
Preview URL
Details

:x:Error

Details

[Error] Run script failed with no detailed information. Please retry building your content again. If the issue still happens, open a ticket in http://SiteHelp and include the error report for our team to troubleshoot

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.

Docs Build status updates of commit 5df32a1:
:x: Validation status: errors
Please follow instructions here which may help to resolve issue.

File
Status
Preview URL
Details

:x:Error

Details

[Error] Run script failed with no detailed information. Please retry building your content again. If the issue still happens, open a ticket in http://SiteHelp and include the error report for our team to troubleshoot

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.

Docs Build status updates of commit 6f80530:
:x: Validation status: errors
Please follow instructions here which may help to resolve issue.

File
Status
Preview URL
Details

:x:Error

Details

:x:Error

Details

[Error] Run script failed with no detailed information. Please retry building your content again. If the issue still happens, open a ticket in http://SiteHelp and include the error report for our team to troubleshoot
[Warning] Cannot find docset information for repository: 6333a4b0-869f-2b4a-a954-6d5820d2e35e and docset name: azuredotnet. Please make sure the docset is correctly provisioned
[Warning] No provisioned docset found in repository 6333a4b0-869f-2b4a-a954-6d5820d2e35e. Please provision or restore the docsets if you want to build them

[Error] Run script failed with no detailed information. Please retry building your content again. If the issue still happens, open a ticket in http://SiteHelp and include the error report for our team to troubleshoot
[Warning] Cannot find docset information for repository: 6333a4b0-869f-2b4a-a954-6d5820d2e35e and docset name: azuredotnet. Please make sure the docset is correctly provisioned
[Warning] No provisioned docset found in repository 6333a4b0-869f-2b4a-a954-6d5820d2e35e. Please provision or restore the docsets if you want to build them

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.


Unblocked site blocked by programm

Operating system / операционная система
windows 11 22h2
Running as service / Запуск программы как сервис
I run it as a regular program / Запускаю программу обычным образом
Describe the bug / Опишите ошибку программы
Site "https://cyberpunk.fandom.com/" does not open with the app, but opens without it.
Additional information / Дополнительная информация
No response

Try add --wrong-chksum то CMD what you use

This helped, thank you.

After a while, the same problem still occurs, and you have to either turn off the application or restart it.


fix: async module runtime for won't run top-level-await

fixes https://github.com/webpack/webpack/issues/16097
Summary

🤖 Generated by Copilot at 6e62cf4
Fix and test a bug in AsyncModuleRuntimeModule that caused an infinite loop for async modules with top-level await and non-existent globals. The fix prevents duplicate execution of normal async modules and the test case uses a fake async module to simulate the bug scenario.
Details

🤖 Generated by Copilot at 6e62cf4

Fix infinite loop bug in async module runtime module when using top-level await with non-existent global variable (link, link)
Add test case for the bug in test/configCases/issues/issue-16097 (link, link, link)

Thank you

Fantastic fix thank you!

Thank you! 🎉


Fixing Node js Spelling Inside the Backend Development Website

hii @kaiwalyakoparkar there is a little spelling mistake inside the backend  development website under the category of build projects by printing the node.js spelling as noge.js.

@PradumTiwari thank you so much for reporting. It will be fixed soon :)

@PradumTiwari this is fixed. Thank yo so much for raising the issue 🙌


[PIE-1385] Update API payload to contain name and version

This PR appends collector name and version to the API payload for the test-collector per PIE-1385

@jimsynz I have a few questions as well as my one about extracting the Version from Buildkite.TestAnalytics.Common.fsproj:

Is there a script to bump the version in the above file that's not just manually changing it in the code?
Does the version in Buildkite.TestAnalytics.Common.fsproj need to match that in Buildkite.TestAnalytics.Xunit

@jimsynz I have a few questions as well as my one about extracting the Version from Buildkite.TestAnalytics.Common.fsproj:
* Is there a script to bump the version in the above file that's not just manually changing it in the code?

I'm not sure I'm afraid. There should be a way to extract the version of the current project but I don't know what it is. I've asked @mbuhot to swing by and see if he knows the answers.

* Does the version in `Buildkite.TestAnalytics.Common.fsproj` need to match that in `Buildkite.TestAnalytics.Xunit`

No, they're separate nuget packages, so they can be versioned independently of each other.

Version = System.Reflection.Assembly.GetExecutingAssembly().GetName().Version.ToString()

Should give you the version number of the Buildkite.TestAnalytics.Common assembly at runtime.

Thanks @mbuhot , @jimsynz are you able to do another review?

@jimsynz are you able to do the release on NuGet? I can't sign up under my buildkite creds and it looks like you're the owner :)

@KatieWright26 0.1.2 has been published.


Implement the new test level features in Summer 15 to deploy triggers faster

A new feature of Summer 15 allows us to specify tests to run when deploying code. It would be great to speed up deployments (and reduce failures) by specifying only the needed test when deploying the trigger. Check out my blog post for details: http://www.verticalcoder.com/2015/06/10/deploy-faster-with-summer-15-features/

I'm using my new Priority tag for this one! :+1:

I've recently written up some shared details on the code coverage issue described in this case, please check the details here and comment at the bottom of the wiki page.

See v1.21 release.


How to test

I wanted to test the bot using your code. In your example you simply have $ composer test. I have no clue how to start this although I got it unto my discord server it doesn't stay.

Running composer test will run PHPUnit to make sure the code does what it's expected to do (though the tests might not be that great, I was never really great at making them).
After setting up the bot it should be on your server but in an offline state. This is because the bot only connects when it sends a notification, it doesn't run all the time because it doesn't need to and doing so would require a script to be running all the time.
If you wanted to actually test the bot's notifications, you'll need an Eloquent model with a routeNotificationForDiscord method that returns the ID of the Discord channel you want to post the notifications to. Once that's setup you can make a new notification that uses the DiscordChannel class in the notification's via method and return a DiscordMessage in the toDiscord method. You can find an example notification in the Usage section of the readme.


Support custom input actions on CesiumCameraController

Fixes #414.
This change adds a set of properties to CesiumCameraController that allows specifying an input action to bind to each action in the camera controller (such as look, move, speed change, etc). This property can specifying a binding for that action, or reference an existing action to use for the input. If no action is specified, CesiumCameraController will create default bindings for that action as it did before this change.

Updated CHANGES.md. In response to your two questions:

InputActionProperty allows the same UI as InputAction, with the addition of allowing InputActionReference assets to be specified in place of a new binding. So I think the name "Input Actions" is still accurate, it will work the same as an InputAction if you want to use it that way.
I don't think the InputActionProperty fields can be set with field initializers. Could possibly be set through the Reset method in the editor? But I don't think Reset runs on existing components, only when you add a new component to the scene, so any camera controllers in scenes people already made before this change would stop working. So we'd need some way of setting these default values on components already added to scenes too.


When the options for filterSelectOptions are variables the component don't rerender

material-react-table version
2.0.0
react & react-dom versions
18.2.0
Describe the bug and the steps to reproduce it
We have a custom filterselectoption for one column. but that filter depends on the values for that same columns.
For example:
We create a new tag, and all tags will be available to filter in that column.

On that image you can see we added a new tag but on filter options don't appear.
I believe it could be related to a memoized component of the multi-select filter.
Minimal, Reproducible Example - (Optional, but Recommended)
      filterSelectOptions: devicesGroups.map((item) => ({         text: item.name,         value: item.name,       })),
Screenshots or Videos (Optional)
Screencast from 03-07-24 09:50:00.webm
Do you intend to try to help solve this bug with your own PR?
No, because I do not know how
Terms

[X] I understand that if my bug cannot be reliably reproduced in a debuggable environment, it will probably not be fixed and this issue may even be closed.

share more code, or even a full sandbox repro. Might just be that your columnDefs did not have the proper dependency array in the useMemo. But impossible to tell if you don't show it.


Error in installation?

Hello. I'm getting this error when trying to run finder--the error occurs with example data as well. Any help would be greatly appreciated.
Traceback (most recent call last):   File "/data/selaginella/finder/finder", line 648, in <module>     main()   File "/data/selaginella/finder/finder", line 586, in main     validateCommandLineArguments(options,logger_proxy,logging_mutex)   File "/data/selaginella/finder/finder", line 235, in validateCommandLineArguments     cmd+=" --runThreadN "+options.cpu TypeError: can only concatenate str (not "int") to str

Hello @vizzerra,
Thank you for checking out finder. Are you working with the latest version which is on GitHub? I had modified the code a few days back which should take of this error. Please issue the command git pull from under the directory where you have downloaded finder. It will update finder with the latest changes. Please let me know if this works.
Thank you.

@sagnikbanerjee15 thank you for your help, but the error still persists (see below). Interestingly, it ran until encountering an error in the metadata file, which I fixed, but the error returned.
May 03 16:47:16 ...... FATAL ERROR, exiting
Traceback (most recent call last):
File "/data/selaginella/finder/finder", line 668, in
main()
File "/data/selaginella/finder/finder", line 629, in main
orchestrateGeneModelPrediction(options,logger_proxy,logging_mutex)
File "/data/selaginella/finder/finder", line 413, in orchestrateGeneModelPrediction
alignReadsAndMergeOutput(options,logger_proxy,logging_mutex)
File "/data/selaginella/finder/scripts/findGenesFromExpression.py", line 307, in alignReadsAndMergeOutput
alignReadsWithSTARRound1(options,Run,options.mrna_md[condition][Run]["Ended"],condition,logger_proxy,logging_mutex)
File "/data/selaginella/finder/scripts/alignReads.py", line 11, in alignReadsWithSTARRound1
cmd+=" --runThreadN "+options.cpu
TypeError: can only concatenate str (not "int") to str

Hi @vizzerra,
Thank you for your response. I have figured out why you are encountering the error and I have fixed the code. Please issues a git pull from your end to get the latest code from GitHub. I am not sure why your system converts the options.cpu into an integer. What system are you running finder on?
Thank you.

That is weird. Finder is running on Linux 4.15 with Ubuntu 18.14.5.
Unfortunately, there is this new error:
File "/data/selaginella/finder/scripts/fileReadWriteOperations.py", line 232, in readAllTranscriptsFromGTFFileInParallel
fhr=open(gtf_filename,"r")
FileNotFoundError: [Errno 2] No such file or directory: '/data/selaginella/DATA/annotations/wallacei_1/assemblies_psiclass_modified/combined/combined.gtf'
Is it not outputting the combined.gtf file or something I should fix on my end? Apologies for the headache!

Hello @vizzerra,
Thank you for reporting this error. I think there might be an issue with the transcript generation. Could you please send me your metadata.csv file? Also, which organism are you trying to annotate? I do not think there is anything wrong in your end. I just want to make sure that the RNA-Seq samples are provided correctly to finder.
Thank you.

Hi. Here is the metadata.csv
BioProject,SRA Accession,Tissues,Description,Date,Read Length (bp),Ended,RNA Seq,process,Location
Dummy,wallacei,leaf,Illumina,5/3/21,100,SE,1,1,~/DATA/annotations/wallacei/wallacei.scafSeq.fastq
Annotating a selaginella genome, a very small vascular plant.

Hi @vizzerra,
I understand the issue. The location needs to be only the directory not the name of the file. The correct content should be:
BioProject,SRA Accession,Tissues,Description,Date,Read Length (bp),Ended,RNA Seq,process,Location
Dummy,wallacei.scafSeq,leaf,Illumina,5/3/21,100,SE,1,1,~/DATA/annotations/wallacei/
finder will reconstruct the whole path by merging the location with the SRA Accession (which in your case is a local file). Hence, it will generate the location as ~/DATA/annotations/wallacei/wallacei.scafSeq.fastq
Hope that helps.
Please let me know if you run into any issues.
Thank you.

Unfortunately, it spits out the same error

Hi @vizzerra,
Oh, I see. Well, that's a bit disappointing. Could you please send me all the data - the RNA-Seq sample, along with the genome sequence and a protein fasta file (if you have one). I will be able to troubleshoot faster if I have access to the data. You can email those to<EMAIL_ADDRESS>Thank you.

Email sent. Thank you.


Use ssheasy as vpn

Hi, my ISP has blocked my server so i can't connect via SSH using Putty
But when i try ssh through ssheasy i have no problem conncting
I am wondering that can i use ssheasy to connect to my server as a vpn(ssh tunnel)

Hi,
ssh tunneling is currently not supported by ssheasy. I had some idea and did some initial work, but it's not easy and currently i don't see when i could work on this.


Update Uniform Fractions Skill Score UFSS in appendixC.rst

In Roberts and Lean 2008, UFSS is defined as 0.5 +􏰆 fo/2 where fo is observed frequency (fraction of observed points exceeding the threshold over the domain)--see Section 2d, halfway down the right column of page 83.  The critical word is observed.
However, the User's Guide v9.0.3 it says UFSS is equivalent to the forecast event proportion--a subtle but important difference if there is any forecast frequency bias. UFSS will differ depending on if it is based on observed event proportion or forecasted event proportion.
Now, I do believe, behind the scenes, MET based the UFSS on the observed frequency, consistent with Roberts and Lean 2008. Only the User's Guide needs to be revised.
I also revised the description of UFSS and formulation to match Roberts and Lean 2008. UFSS is simply halfway between perfect skill (unity) and a random FSS (equal to observed event proportion).

I removed the reference to UFSS needing to be exceeded in order to have skill. It depends on your baseline. Roberts and Lean (2008) offer FSSrandom as a possible baseline, based on a random forecast FSS. It is only equal to the total observed event proportion. For rare events, this is a much lower "skill threshold" than UFSS.


Migrate DiaperBase to inventory.diaper.app, but leave static content at diaper.app

This is possible right? I'm not sure how complicated this would be (particularly with extracting the static pages). The static page "sign-in" links would need to be adjusted to point to the new URL.

I think we need a whole refactoring of the static content.

[ ] Migrate it from foundation to bootstrap 4 for consistency with the rest of the app
[ ] Add more content

[ ] About page?
[ ] Contact page?

[ ] Partner sign in link (that goes to partner app)
[ ] Diaper bank sign in link (that goes to inventory?)

Closing this because we've since talked about ideas that would require the static content have ready access to some of the application data.


Idea: Indefinite end?

In the UI at the moment there is no way to specify that the recurrence should occur indefinitely.

Is that feasible?

It's published.
Thanks for the pull request !