Skipped mzXML files generate missing sample errors

ISSUE SUBMITTER SECTION
Problem Description
When an mzXML file's path has been supplied in the mzXML file column, and the row is marked as "skip", the load results in an AllMissingSamples error, which should not happen.  If the file is skipped, then the sample lookup should not occur (meaning, it should not buffer a RecordDoesNotExist error and that those should not be summarized in MissingSamples errors or AllMissingSamples errors).
Note that the error can be avoided by simply not entering anything in the mzXML file column, and putting the file's name minus extension in the sample header column (I think - I did not verify that the headers were actually present, although I think they're not there, because I recall generating a template and those rows were absent).
Steps to reproduce
This currently happens when attempting to load the Perturbative Infusions study.
Current behavior
An AllMissingSamples error occurs (is buffered), listing these 2 samples:

prerun
prerun_20240923175535

The exception is buffered inside the leftover mzXMLs conditional, when the file's basename without the extension is used to search the Sample table.
Expected behavior
Since the rows exist and had a skip value in the skip column, those samples should not have been identified as either leftovers or missing samples and thus, no error or warning should have been buffered about them.
Suggested Change

None provided
Comment

None

ISSUE OWNER SECTION

Assumptions

None provided
Limitations

None provided
Affected Components

None provided
Requirements

None provided
DESIGN
GUI Change description

None provided
Code Change Description

None provided
Tests

None provided

Discovered a few related issues:

Upon rollback, not all files are removed

It appears that files with dashes (or skipped files?) are not removed
It's probably "skipped" files that aren't removed, because (one occurrence?) prerun.mzXML is present

Skipped blanks are not skipped
There's only 1 archive file named prerun.mzXML even though there are about a dozen of them, so I'm not sure the archive file creation is working correctly, though it could be that all other versions were removed

All such files were last modified yesterday, so the load script isn't "getting" them
None of the prerun files were marked as "removed" due to rollback, which suggests they weren't created, and it appears that all other occurrences of skipped files were marked as removed (with the hash components in the names), so it looks like it's skipping the ones without the hash components. I'm not entirely sure what's going on.

Some more debug info:
(tracebase) [tracebase@tb9-dev tracebase]$ find ~/incoming/perturbative_infusions/ -name prerun.mzXML
/home/tracebase/incoming/perturbative_infusions/e480_wlu_Unknown_20240712-Trp-Acet-tracing/prerun.mzXML
/home/tracebase/incoming/perturbative_infusions/e480_wlu_Unknown_20240712-Trp-Acet-tracing/scan2/prerun.mzXML
/home/tracebase/incoming/perturbative_infusions/e480_wlu_Unknown_20240712-Trp-Acet-tracing/scan3/prerun.mzXML
/home/tracebase/incoming/perturbative_infusions/e480_wlu_241016_Pro-Lin/prerun.mzXML
/home/tracebase/incoming/perturbative_infusions/e480_wlu_241016_Pro-Lin/scan2/prerun.mzXML
/home/tracebase/incoming/perturbative_infusions/e480_wlu_241016_Pro-Lin/scan3/prerun.mzXML
/home/tracebase/incoming/perturbative_infusions/e480_wlu_20240909_Val-Ile/prerun.mzXML
/home/tracebase/incoming/perturbative_infusions/e480_wlu_20240909_Val-Ile/scan2/prerun.mzXML
/home/tracebase/incoming/perturbative_infusions/e480_wlu_20240909_Val-Ile/scan3/prerun.mzXML
/home/tracebase/incoming/perturbative_infusions/e480_ricardo_acetate-M28 Citrate-M29 mouse infusions 15Aug24/prerun.mzXML
/home/tracebase/incoming/perturbative_infusions/e480_ricardo_acetate-M28 Citrate-M29 mouse infusions 15Aug24/scan2/prerun.mzXML
/home/tracebase/incoming/perturbative_infusions/e480_ricardo_acetate-M28 Citrate-M29 mouse infusions 15Aug24/scan3/prerun.mzXML
/home/tracebase/incoming/perturbative_infusions/e480_wlu_20240603-glc-tracing/prerun.mzXML
/home/tracebase/incoming/perturbative_infusions/e480_wlu_20240603-glc-tracing/scan2/prerun.mzXML
/home/tracebase/incoming/perturbative_infusions/e480_wlu_20240603-glc-tracing/scan3/prerun.mzXML
/home/tracebase/incoming/perturbative_infusions/e480_wlu_20240923_Lys-Leu/prerun.mzXML
/home/tracebase/incoming/perturbative_infusions/e480_wlu_20240923_Lys-Leu/scan2/prerun.mzXML
/home/tracebase/incoming/perturbative_infusions/e480_wlu_20240923_Lys-Leu/scan3/prerun.mzXML
/home/tracebase/incoming/perturbative_infusions/e480_wlu_20240618-Arg-Cys-tracing/prerun.mzXML
/home/tracebase/incoming/perturbative_infusions/e480_wlu_20240618-Arg-Cys-tracing/scan2/prerun.mzXML
/home/tracebase/incoming/perturbative_infusions/e480_wlu_20240618-Arg-Cys-tracing/scan3/prerun.mzXML
(tracebase) [tracebase@tb9-dev tracebase]$ find ~/incoming/perturbative_infusions/ -name acet-pos-M19-T24.mzXML
/home/tracebase/incoming/perturbative_infusions/e480_wlu_Unknown_20240712-Trp-Acet-tracing/acet-pos-M19-T24.mzXML
/home/tracebase/incoming/perturbative_infusions/e480_wlu_Unknown_20240712-Trp-Acet-tracing/scan2/acet-pos-M19-T24.mzXML
/home/tracebase/incoming/perturbative_infusions/e480_wlu_Unknown_20240712-Trp-Acet-tracing/scan3/acet-pos-M19-T24.mzXML
(tracebase) [tracebase@tb9-dev tracebase]$ md5sum `find ~/incoming/perturbative_infusions/ -name prerun.mzXML`
9f860f2224364ed70496452d1488e97b  /home/tracebase/incoming/perturbative_infusions/e480_wlu_Unknown_20240712-Trp-Acet-tracing/prerun.mzXML
9203099135ce61d0e889d80db0a97c35  /home/tracebase/incoming/perturbative_infusions/e480_wlu_Unknown_20240712-Trp-Acet-tracing/scan2/prerun.mzXML
9203099135ce61d0e889d80db0a97c35  /home/tracebase/incoming/perturbative_infusions/e480_wlu_Unknown_20240712-Trp-Acet-tracing/scan3/prerun.mzXML
c842a349225759d11af793683b7144c5  /home/tracebase/incoming/perturbative_infusions/e480_wlu_241016_Pro-Lin/prerun.mzXML
0f93d7fed02256bad6acdbb64b6411d1  /home/tracebase/incoming/perturbative_infusions/e480_wlu_241016_Pro-Lin/scan2/prerun.mzXML
0f93d7fed02256bad6acdbb64b6411d1  /home/tracebase/incoming/perturbative_infusions/e480_wlu_241016_Pro-Lin/scan3/prerun.mzXML
eabee2063644cce40d48130f348506c5  /home/tracebase/incoming/perturbative_infusions/e480_wlu_20240909_Val-Ile/prerun.mzXML
5a09b2e37cbfa0be5e0cc7c2c241266d  /home/tracebase/incoming/perturbative_infusions/e480_wlu_20240909_Val-Ile/scan2/prerun.mzXML
5a09b2e37cbfa0be5e0cc7c2c241266d  /home/tracebase/incoming/perturbative_infusions/e480_wlu_20240909_Val-Ile/scan3/prerun.mzXML
md5sum: /home/tracebase/incoming/perturbative_infusions/e480_ricardo_acetate-M28: No such file or directory
md5sum: Citrate-M29: No such file or directory
md5sum: mouse: No such file or directory
md5sum: infusions: No such file or directory
md5sum: 15Aug24/prerun.mzXML: No such file or directory
md5sum: /home/tracebase/incoming/perturbative_infusions/e480_ricardo_acetate-M28: No such file or directory
md5sum: Citrate-M29: No such file or directory
md5sum: mouse: No such file or directory
md5sum: infusions: No such file or directory
md5sum: 15Aug24/scan2/prerun.mzXML: No such file or directory
md5sum: /home/tracebase/incoming/perturbative_infusions/e480_ricardo_acetate-M28: No such file or directory
md5sum: Citrate-M29: No such file or directory
md5sum: mouse: No such file or directory
md5sum: infusions: No such file or directory
md5sum: 15Aug24/scan3/prerun.mzXML: No such file or directory
7ed92c8f740a69074b5fa2cc59d121a1  /home/tracebase/incoming/perturbative_infusions/e480_wlu_20240603-glc-tracing/prerun.mzXML
e81c91883a2309cc04225a647d61fcee  /home/tracebase/incoming/perturbative_infusions/e480_wlu_20240603-glc-tracing/scan2/prerun.mzXML
e81c91883a2309cc04225a647d61fcee  /home/tracebase/incoming/perturbative_infusions/e480_wlu_20240603-glc-tracing/scan3/prerun.mzXML
c89ea9bc792447a41b9f623a28a9c0d0  /home/tracebase/incoming/perturbative_infusions/e480_wlu_20240923_Lys-Leu/prerun.mzXML
9e358f6fd8307179d0086d5d64016639  /home/tracebase/incoming/perturbative_infusions/e480_wlu_20240923_Lys-Leu/scan2/prerun.mzXML
9e358f6fd8307179d0086d5d64016639  /home/tracebase/incoming/perturbative_infusions/e480_wlu_20240923_Lys-Leu/scan3/prerun.mzXML
3a3851051de7cc3080bb1dfbf5dec29e  /home/tracebase/incoming/perturbative_infusions/e480_wlu_20240618-Arg-Cys-tracing/prerun.mzXML
e96d0de7e0dbadff5b21e3c929133ef4  /home/tracebase/incoming/perturbative_infusions/e480_wlu_20240618-Arg-Cys-tracing/scan2/prerun.mzXML
e96d0de7e0dbadff5b21e3c929133ef4  /home/tracebase/incoming/perturbative_infusions/e480_wlu_20240618-Arg-Cys-tracing/scan3/prerun.mzXML
(tracebase) [tracebase@tb9-dev tracebase]$ find ~/incoming/perturbative_infusions/ -name "acet-pos-M19-T24*"
/home/tracebase/incoming/perturbative_infusions/e480_wlu_Unknown_20240712-Trp-Acet-tracing/acet-pos-M19-T24.mzXML
/home/tracebase/incoming/perturbative_infusions/e480_wlu_Unknown_20240712-Trp-Acet-tracing/scan2/acet-pos-M19-T24.mzXML
/home/tracebase/incoming/perturbative_infusions/e480_wlu_Unknown_20240712-Trp-Acet-tracing/scan3/acet-pos-M19-T24.mzXML
(tracebase) [tracebase@tb9-dev tracebase]$ find / -name "acet-pos-M19-T24*"
afs/                   boot/                  etc/                   lib/                   media/                 opt/                   root/                  sbin/                  swapfile               tmp/                   tracebase-staging/     usr/
bin/                   dev/                   home/                  lib64/                 mnt/                   proc/                  run/                   srv/                   sys/                   tracebasedev-archive/  tracebase-submissions/ var/
(tracebase) [tracebase@tb9-dev tracebase]$ find /tracebasedev-archive/archive/archive_files -name "acet-pos-M19-T24*"
archive_files/     archive_files.old/
(tracebase) [tracebase@tb9-dev tracebase]$ find /tracebasedev-archive/archive/archive_files/2024-12 -name "acet-pos-M19-T24*"
/tracebasedev-archive/archive/archive_files/2024-12/ms_data/acet-pos-M19-T24.mzXML

I think I found at least part of the issue.  When the user enters paths in the mzxml file column in the peak annotation details sheet, that path and the path of the file provided are assumed to either both be relative or both be absolute.  When they're not, the mzxml metadata is not obtained.
That relates to the sample being "missing", because it assumes that, since it cannot associate the file(s) with any rows, that it is totally unaccounted for, so it searches for that sample.  Of course, it doesn't find it, because that sample was skipped.

OK, I have fixed the missing sample issue.

[ ] Create a summarization of MzXMLSkipRowError
[ ] Make sure that files are removed from the archive

[ ] All NoScans exceptions say they come from row 2, which is wrong:

EXCEPTION1(WARNING): AllNoScans: The following mzXML files, obtained from the indicated file locations, were not loaded because they contain no scan data:
	sheet [Peak Annotation Details] in study doc_241126.xlsx:
		/tracebase-staging/incoming/perturbative_infusions/e480_wlu_Unknown_20240712-Trp-Acet-tracing/scan2/trp-neg-M17-T03.mzXML found on row(s): ['2']
		/tracebase-staging/incoming/perturbative_infusions/e480_wlu_Unknown_20240712-Trp-Acet-tracing/scan2/trp-neg-M17-T22-6h.mzXML found on row(s): ['2']
		/tracebase-staging/incoming/perturbative_infusions/e480_wlu_Unknown_20240712-Trp-Acet-tracing/scan2/acet-neg-M18-T24.mzXML found on row(s): ['2']
		/tracebase-staging/incoming/perturbative_infusions/e480_wlu_Unknown_20240712-Trp-Acet-tracing/scan2/acet-neg-M18-T22-6h.mzXML found on row(s): ['2']
		/tracebase-staging/incoming/perturbative_infusions/e480_wlu_Unknown_20240712-Trp-Acet-tracing/scan2/acet-neg-M19-T18.mzXML found on row(s): ['2']
		/tracebase-staging/incoming/perturbative_infusions/e480_wlu_Unknown_20240712-Trp-Acet-tracing/scan2/trp-neg-M16-T03.mzXML found on row(s): ['2']
		/tracebase-staging/incoming/perturbative_infusions/e480_wlu_Unknown_20240712-Trp-Acet-tracing/scan2/trp-neg-M16-T07.mzXML found on row(s): ['2']
		/tracebase-staging/incoming/perturbative_infusions/e480_wlu_Unknown_20240712-Trp-Acet-tracing/scan2/acet-neg-M18-T12.mzXML found on row(s): ['2']
		/tracebase-staging/incoming/perturbative_infusions/e480_wlu_Unknown_20240712-Trp-Acet-tracing/scan2/trp-neg-M16-T25.mzXML found on row(s): ['2']

...

[ ] AggregatedErrors exception raised when all exceptions are warnings and not in validate mode

  File "/var/www/tracebase/manage.py", line 22, in <module>
    main()
  File "/var/www/tracebase/manage.py", line 18, in main
    execute_from_command_line(sys.argv)
  File "/var/www/tracebase/DataRepo/management/commands/load_table.py", line 420, in handle_wrapper
    self.saved_aes.buffer_error(e)
  File "/var/www/tracebase/DataRepo/utils/exceptions.py", line 2466, in buffer_error
    return self.buffer_exception(
  File "/var/www/tracebase/DataRepo/utils/exceptions.py", line 2416, in buffer_exception
    buffered_tb_str = self.get_buffered_traceback_string()
  File "/var/www/tracebase/DataRepo/utils/exceptions.py", line 2396, in get_buffered_traceback_string
    for step in traceback.format_stack()

The above caught exception had a partial traceback:

  File "/var/www/tracebase/DataRepo/management/commands/load_table.py", line 409, in handle_wrapper
    retval = fn(self, *args, **options)
  File "/var/www/tracebase/DataRepo/management/commands/load_study.py", line 157, in handle
    self.load_data()
  File "/var/www/tracebase/DataRepo/management/commands/load_table.py", line 450, in load_data
    return self.loader.load_data(*args, **kwargs)
  File "/var/www/tracebase/DataRepo/loaders/base/table_loader.py", line 2376, in load_wrapper
    raise aes_set
  File "/var/www/tracebase/DataRepo/loaders/base/table_loader.py", line 2247, in load_wrapper
    retval = fn(*args, **kwargs)
  File "/var/www/tracebase/DataRepo/models/maintained_model.py", line 1245, in wrapper
    return fn(*args, **kwargs)
  File "/var/www/tracebase/DataRepo/loaders/study_loader.py", line 556, in load_data
    raise self.load_statuses.get_final_exception()
EXCEPTION1(ERROR): AggregatedErrorsSet: 1 categories had exceptions, including 0 in an error state and 1 in a warning state.
study doc_241126.xlsx: AggregatedErrors Summary (0 errors / 3 warnings):
	EXCEPTION1(WARNING): AllNoScans: The following mzXML files, obtained from the indicated file locations, were not loaded because they contain no scan data:

...

Traceback (most recent call last):
  File "/var/www/tracebase/manage.py", line 22, in <module>
    main()
  File "/var/www/tracebase/manage.py", line 18, in main
    execute_from_command_line(sys.argv)
  File "/usr/local/tracebase/lib/python3.9/site-packages/django/core/management/__init__.py", line 442, in execute_from_command_line
    utility.execute()
  File "/usr/local/tracebase/lib/python3.9/site-packages/django/core/management/__init__.py", line 436, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/usr/local/tracebase/lib/python3.9/site-packages/django/core/management/base.py", line 412, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/usr/local/tracebase/lib/python3.9/site-packages/django/core/management/base.py", line 458, in execute
    output = self.handle(*args, **options)
  File "/var/www/tracebase/DataRepo/management/commands/load_table.py", line 427, in handle_wrapper
    raise self.saved_aes
DataRepo.utils.exceptions.AggregatedErrors: 1 exceptions occurred, including type(s): [AggregatedErrorsSet].
AggregatedErrors Summary (1 errors / 0 warnings):
	EXCEPTION1(ERROR): AggregatedErrorsSet: 1 categories had exceptions, including 0 in an error state and 1 in a warning state.
study doc_241126.xlsx: AggregatedErrors Summary (0 errors / 3 warnings):
	EXCEPTION1(WARNING): AllNoScans: The following mzXML files, obtained from the indicated file locations, were not loaded because they contain no scan data:

...


Property Replacement Quirks

Property replacement does not always work if the replaceable text is not wrapped in tags properly.
This does not work:
(div id="test"){{firstname}} {{lastname}}(/div)
This does:
(div id="test")(span){{firstname}}(/span) (span){{lastname}}(/span)(/div)
Without the tags immediately surrounding the text to be replaced, it does not work.  Even putting a bit of text between the two properties does not resolve the issue:
(div id="test"){{firstname}} testing {{lastname}}(/div)

This is as designed.
https://www.polymer-project.org/1.0/docs/devguide/data-binding.html#binding-to-text-content
"The binding annotation must currently span the entire content of the tag ... String concatenation is not supported inside a tag, and the tag can’t contain any whitespace".

Mystery solved.
Thanks!


Race/double build in Windows installer build step

2019-02-02T15:51:13.3322364Z C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\Common7\IDE\VC\VCTargets\Microsoft.CppCommon.targets(720,5): warning MSB6008: Forcing a rebuild of all sources due to an error with the tracking logs. The process cannot access the file 'F:\vsagent\3\s\src\Servers\IIS\AspNetCoreModuleV2\OutOfProcessRequestHandler\Release\OutOfPro.7F87406C.tlog\link-rc.read.1.tlog' because it is being used by another process. [F:\vsagent\3\s\src\Servers\IIS\AspNetCoreModuleV2\OutOfProcessRequestHandler\OutOfProcessRequestHandler.vcxproj]
2019-02-02T15:51:13.3322550Z C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\Common7\IDE\VC\VCTargets\Microsoft.CppCommon.targets(720,5): error MSB6003: The specified task executable "link.exe" could not be run. The process cannot access the file 'F:\vsagent\3\s\src\Servers\IIS\AspNetCoreModuleV2\OutOfProcessRequestHandler\Release\OutOfPro.7F87406C.tlog\link-rc.read.1.tlog' because it is being used by another process. [F:\vsagent\3\s\src\Servers\IIS\AspNetCoreModuleV2\OutOfProcessRequestHandler\OutOfProcessRequestHandler.vcxproj]
2019-02-02T15:51:13.3322653Z     1 Warning(s)
2019-02-02T15:51:13.3322697Z     1 Error(s)

https://dev.azure.com/dnceng/public/_build/results?buildId=85876
https://dev.azure.com/dnceng/public/_build/results?buildId=86710
cc @natemcmaster

@natemcmaster seems like projects are duplicated in ProjectsToBuild collection:

Look at the metadata. They should differ by Platform.


undefined issue

I tried to implement this project https://github.com/Tallyb/cucumber-playwright on my computer but I fixed all config issues I did not fix the undefined issue because this project does not use glue so How can I fix this problem? Thanks.

Do you have a cucumber extension installed?

Hi Tally,
Yes it installed.

As you can see on the CI, it is working ok. The one thing I can think of - what is your node version? the cucumber configuration is in mjs file, so the minimum should be node 14.


Deal with trailing slash on predict url

Deal with trailing slash on predict url
querying
localhost:1337/pong/predict/
should behave the same as
localhost:1337/pong/predict

@simon-mo Is it possible in the current version?

Merged with #721


wiring issue

Could you maybe give actual pin numbers on the tft panel in the following diagram?
http://prntscr.com/smmjlo
The ESP32 device I have and the nice ESP32_ Pinout diagram do not match at all. As well, the wiring pictures seem to be slightly out of sync compared to the diagrams. for example, the brown wire on the TFT picture goes to a NON ground pin on the ESP32.. Whereas the green wire, is hooked up to a "ground" on the ESP32, is not hooked to a ground on the TFT.

The picture with the wiring relates to a ESP32 DoIt Dev kit.
I think I will remove the pictures since they can cause more confusion than help clarifying the issue. There are just too many different board layouts for ESP32 dev kits.
I'll update the table as well to match with the ER-TFT module. Check out page 11 on the docs for details till then.

Updated README


Feature request - Batch import of Imported Geometry

If multiple files are selected for import, then it would be convenient if we had to choose import settings just once and have those settings be applied to all selected models.

Wrong repository. Ask in https://github.com/MontyTRC89/Tomb-Editor/issues


Do Not Merge. [BEAM-5515] fix beam java sdk nightly snapshot

Please add a meaningful description for your change here

Follow this checklist to help us incorporate your contribution quickly and easily:

[ ] Format the pull request title like [BEAM-XXX] Fixes bug in ApproximateQuantiles, where you replace BEAM-XXX with the appropriate JIRA issue, if applicable. This will automatically link the pull request to the issue.
[ ] If this contribution is large, please file an Apache Individual Contributor License Agreement.

It will help us expedite review of your Pull Request if you tag someone (e.g. @username) to look at it.


removed gdal

This PR is to see which libraries I can remove from minimal-conda-requirements.txt without breaking the iris tests.
I will make one change per PR, which will be described in the commit message.

I will make one change per PR

Since it doesn't look like you're doing this in parallel, you could push one commit at a time and just wait for the CI.

obsoleted by #2199


Change namespaces to be compliant with folder structure

There is a need to change namespaces in the app solution to be compliant with folder structure (except DB classes).
The app becomes bigger and often I see in PRs new classes with incorrect for current approach namespaces. To avoid such fixes I assume it's time to start use the most correct approach of namespace using.

This adheres to point 20 in https://starcounter.gitbooks.io/guidelines/content/coding_style/csharp_code_formatting.html

Closing as done.


Draft pr combining pfm and reece's cleanup for v13

Closing since PFM is in main, and now in my PR which works as well


3.0.0.M1 -> Grails 3.0.7 and 3.0.8 -> Can not apply staticRules to REST resource URLs

For a simple working example, view
https://github.com/codeconsole/grails3bugs
Run the app and try to view
http://localhost:8080/cars.json
which has the following staticRules:
	'/car/**':          ['permitAll'],
	'/cars.*':          ['permitAll']

You will be redirected to
/login/auth

See explanation of the cause of this problem here https://github.com/grails/grails-core/issues/9343#issuecomment-147637671
The issue is that the configuration:
 '/cars.*':          ['permitAll']

Becomes a nested map such as [cars:['*':['permitAll']]]. I don't believe we will be fixing this in core, the plugin will need to compensate.
Really as I said the plugin needs to be updated to use the new configuration APIs in Grails 3, as it is still using manual merging and Grails 2.x techniques via ConfigObject etc.

@graemerocher  Could you open "the new configuration APIs in Grails 3"? Reading through security plugin, I can see what you mean by manual merging. There seems to be a good potential for simplification. I also read grails documentation on plugins but could not really see what the new API is.
I would like to attempt to help in such a simplification using the right methods, since I believe many people need this plugin to be in good shape. As things stand, the plugin at Milestone1 breaks a 3.1.0.M2 grails application. I want to upgrade to 3.1.0.M2 due to problems with 3.0.9, but cannot because of this.

Defaults should go into grails-app/conf/plugin.yml
See
https://github.com/grails/grails-core/blob/master/grails-core/src/main/groovy/grails/config/Config.groovy
example of usage here https://github.com/grails/grails-core/blob/master/grails-plugin-controllers/src/main/groovy/org/grails/plugins/web/controllers/ControllersGrailsPlugin.groovy#L71
use string keys, second arguments i the type, third argument is the default value
Getting the value will be
  config.getProperty(“grails.plugin.springsecurity.controllerAnnotations.staticRules", Map, [:])

and if there value is a List it is a rule definitions, if it is another map, then you will have to re-add the “." and traverse through the nested values

Thanks for the info. A good starting point.

This works in 3.0.0.M2, but you need to change a couple of things in the test app. The core fix is to update controllerAnnotations.staticRules to the newer format described in the docs:
grails.plugin.springsecurity.controllerAnnotations.staticRules = [
   [pattern: '/',               access: 'permitAll'],
   [pattern: '/error',          access: 'permitAll'],
   [pattern: '/index',          access: 'permitAll'],
   [pattern: '/index.gsp',      access: 'permitAll'],
   [pattern: '/shutdown',       access: 'permitAll'],
   [pattern: '/assets/**',      access: 'permitAll'],
   [pattern: '/**/js/**',       access: 'permitAll'],
   [pattern: '/**/css/**',      access: 'permitAll'],
   [pattern: '/**/images/**',   access: 'permitAll'],
   [pattern: '/**/favicon.ico', access: 'permitAll'],

   [pattern: '/car/**', access: 'ROLE_ADMIN'],
   [pattern: '/car.*',  access: 'ROLE_ADMIN']
]

Additionally note that both mappings require 'car' since the controller is what's guarded, independent of url mappings.
You also need to update BootStrap to assign ROLE_ADMIN to the test user.


Update powerplatformlicensingforcitizendeveloper.md

added content disclaimer - this has been flagged as needing a disclaimer
Please use this template to ensure easier processing of your pull request
Category

[x ] Content fix
[ ] New article

Docs Build status updates of commit 4d89cff:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

Community/powerplatformlicensingforcitizendeveloper.md
:white_check_mark:Succeeded
View

For more details, please refer to the build report.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
For any questions, please:Try searching the docs.microsoft.com contributor guidesPost your question in the Docs support channel


Run tests directly on GA runner

Signed-off-by: Patrik Stas<EMAIL_ADDRESS>

Codecov Report

Merging #622 (b338758) into main (4517ef7) will decrease coverage by 11.42%.
The diff coverage is 47.08%.

@@             Coverage Diff             @@
##             main     #622       +/-   ##
===========================================
- Coverage   58.91%   47.49%   -11.43%
===========================================
  Files         200      199        -1
  Lines       19273    14346     -4927
  Branches     4385     2778     -1607
===========================================
- Hits        11354     6813     -4541
- Misses       3564     5324     +1760
+ Partials     4355     2209     -2146

Flag
Coverage Δ

integration-aries-vcx
?

unittests-agency-client
29.12% <ø> (ø)

unittests-aries-vcx
47.02% <47.08%> (-0.28%)
:arrow_down:

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files
Coverage Δ

...s_vcx/src/indy/primitives/credential_definition.rs
0.00% <0.00%> (-44.40%)
:arrow_down:

aries_vcx/src/utils/test_logger.rs
6.81% <ø> (ø)

aries_vcx/tests/utils/scenarios.rs
0.00% <ø> (-71.14%)
:arrow_down:

aries_vcx/src/handlers/issuance/issuer.rs
51.55% <33.33%> (-6.36%)
:arrow_down:

...vcx/src/protocols/issuance/holder/state_machine.rs
59.76% <44.52%> (-5.52%)
:arrow_down:

aries_vcx/src/handlers/issuance/holder.rs
50.81% <50.00%> (-14.11%)
:arrow_down:

...vcx/src/protocols/issuance/issuer/state_machine.rs
65.30% <51.61%> (-5.23%)
:arrow_down:

...tocols/proof_presentation/prover/states/initial.rs
0.00% <0.00%> (-100.00%)
:arrow_down:

aries_vcx/src/handlers/discovery/mod.rs
0.00% <0.00%> (-87.10%)
:arrow_down:

aries_vcx/src/indy/test_utils.rs
0.00% <0.00%> (-82.94%)
:arrow_down:

... and 110 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.


fix: track jobItemWorkReq function time

Motivation
The current metric for jobItemWorkReq does not match cpu profile
Description

Track time for signature deserialization in histogram instead of gauge, for now we mainly care about the total time used for this api per epoch to match the profile. When investigate any issues we can count on le too
Same for pubkey aggregation

Closes #6200

:tada: This PR is included in v1.14.0 :tada:


Error in wiki about key word confidence in function locateOnScreen

In the english wiki,it said there's a key word in function locateOnScreen named confidance. So the usage of this function can be button7location = pyautogui.locateOnScreen('calc7key.png', confidence=0.9).
But in the file pyautogui/screenshotUtil.py, the function is locateOnScreen(image, grayscale=False,region=None):, and of course I can't use such a key word.

I installed opencv_contrib_python-<IP_ADDRESS>-cp35-cp35m-win_amd64 with python 3.5.
Then i import numpy,pyautogui,cv2 and use the command :
pyautogui.click('st.png',confidence=0.9)
i get the following error:
Traceback (most recent call last):
File "", line 1, in
TypeError: click() got an unexpected keyword argument 'confidence'
Please help.

Same here.

Same here


[Meshery UI] Show only installed adapters on system dashboard

Current Behavior
In System Dashboard the adapter section is showing all adapters including active and inactive.
Desired Behavior
Only installed adapters by the user should be visible on the system dashboard.
Implementation
Configuring shows the adapters component in the dashboard UI such that it sorts adapters and shows only the installed adapters.
Mockups

Contributor Guide

@leecalcote If my implementation approach is going in the wrong direction, please correct me.

@Chadha93 thats right. Any unregistered adapters should not have a corresponding menu item for its respective service mesh type.

@Chadha93 : As a part of this issue , the left side pane also should show the service Mesh that is relevant for an organisation .
For Example : if i have Istio in my infra, then I should be able to see only ISTIO mesh on the left side pane.

@Chadha93 : As a part of this issue , the left side pane also should show the service Mesh that is relevant for an organisation .
For Example : if i have Istio in my infra, then I should be able to see only ISTIO mesh on the left side pane.

In the navigator itself ?

@Chadha93 : As a part of this issue , the left side pane also should show the service Mesh that is relevant for an organisation .
For Example : if i have Istio in my infra, then I should be able to see only ISTIO mesh on the left side pane.

In the navigator itself ?

yes in the , left pane of the meshery , it shows all the service mesh, which should be depending upon the service mesh that is installed in my K8S cluster. You can get this info may be from the kube config or some cloud provider config file (though i am not sure) but it should be part of some config file

Any updates on this issue ?

@Chadha93 has this issue been solved?

@Chadha93 has this issue been solved?

Nope, not finding a proper context of the solution. @warunicorn19 would like to take this up?

@Chadha93, don't know how much ill be able to get done, but I'll take a look 😅

will deal with this when we switch to state management.

I have made a fix for this issue, please check if it is the best approach.
New to this project, looking to forward to know about better approaches or feedbacks if any.


Make code compatible with ES5 strict mode

In version v2.0.0, an error with the word "Uncaught TypeError: 'caller', 'callee', and 'arguments' properties may not be accessed on strict mode functions or the arguments objects for calls to them" appears.
I try to fix the problem in the https://github.com/kuankuan2007/particles.js/tree/kuankuan2007

Hi @kuankuan2007, ran into the same issue since arguments.callee within the deepExtend declaration is deprecated and the strict mode doesn't like that. That being said, this should fix it:
Object.deepExtend = function(destination, source) {
    function extendRecursive(dest, src) {
        for (var property in src) {
            if (src.hasOwnProperty(property)) {
                if (src[property] && src[property].constructor && src[property].constructor === Object) {
                    dest[property] = dest[property] || {};
                    extendRecursive(dest[property], src[property]);
                } else {
                    dest[property] = src[property];
                }
            }
        }
    }

    extendRecursive(destination, source);
    return destination;
`};

Yes, this is a very good solution.


[BACKLOG-8210] Fixed issues with send-true and send-false step fields

Build Completed
:x: This pull request has errors. They will need to be addressed before it can be accepted. See below for more details. Some links are also available below for further assistance in addressing those issues.
Build Commands
ant -Dtestreports.dir=bin/reports/unit-test -f engine/build.xml clean-all resolve jacoco && ant -Dtestreports.dir=bin/reports/integration-test -f engine/build.xml jacoco-integration checkstyle publish-local

Cleanup Commands
rm -r ~/.ivy2/local || echo no publish local to remove

Changed files
  engine/src/org/pentaho/di/trans/steps/filterrows/FilterRowsMeta.java

Unit test coverage change
These statistics help you identify how your changes have affected the coverage of the following files. If a file is not in this list, then its coverage was not affected by your changes. To get some help interpreting these metrics, please refer to Jacoco's documentation.
org.pentaho.di.job.entries.evaluatetablecontent.JobEntryEvalTableContent

Branch Change: -1.01%:small_red_triangle_down:
Complexity Change: -1.06%:small_red_triangle_down:
Instruction Change: -.17%:small_red_triangle_down:

org.pentaho.di.job.entries.ftpsget.JobEntryFTPSGet

Branch Change: -.53%:small_red_triangle_down:
Complexity Change: -.56%:small_red_triangle_down:
Instruction Change: -.07%:small_red_triangle_down:

org.pentaho.di.job.entries.waitforsql.JobEntryWaitForSQL

Branch Change: -.66%:small_red_triangle_down:
Complexity Change: -.93%:small_red_triangle_down:
Instruction Change: -.13%:small_red_triangle_down:

org.pentaho.di.trans.step.StepMeta

Instruction Change: -.19%:small_red_triangle_down:
Line Change: -.27%:small_red_triangle_down:

org.pentaho.di.trans.steps.combinationlookup.CombinationLookupMeta

Instruction Change: -.04%:small_red_triangle_down:

org.pentaho.di.trans.steps.constant.ConstantMeta

Instruction Change: -.10%:small_red_triangle_down:

org.pentaho.di.trans.steps.dbproc.DBProcMeta

Instruction Change: -.11%:small_red_triangle_down:

org.pentaho.di.trans.steps.dimensionlookup.DimensionLookupMeta

Branch Change: -.66%:small_red_triangle_down:
Complexity Change: -.41%:small_red_triangle_down:
Instruction Change: -.50%:small_red_triangle_down:
Line Change: -.24%:small_red_triangle_down:

org.pentaho.di.trans.steps.filterrows.FilterRowsMeta

Branch Change:  + 1.56%
Complexity Change:  + 1.67%
Instruction Change:  + 1.46%
Line Change:  + 1.25%

org.pentaho.di.trans.steps.groupby.GroupByMeta

Branch Change:  + 4.17%
Complexity Change:  + 4.21%
Instruction Change:  + .28%
Line Change:  + .75%

org.pentaho.di.trans.steps.ifnull.IfNullMeta

Branch Change:  + 2.00%
Complexity Change:  + 1.89%
Instruction Change:  + .17%

org.pentaho.di.trans.steps.memgroupby.MemoryGroupByMeta

Branch Change:  + 3.53%
Complexity Change:  + 4.05%
Instruction Change:  + .18%
Line Change:  + .48%

org.pentaho.di.trans.steps.replacestring.ReplaceStringMeta

Instruction Change: -.07%:small_red_triangle_down:

org.pentaho.di.trans.steps.rowgenerator.RowGeneratorMeta

Instruction Change: -.08%:small_red_triangle_down:

org.pentaho.di.trans.steps.setvalueconstant.SetValueConstantMeta

Instruction Change: -.14%:small_red_triangle_down:

org.pentaho.di.trans.steps.tableoutput.TableOutputMeta

Branch Change: -.91%:small_red_triangle_down:
Complexity Change: -.82%:small_red_triangle_down:

org.pentaho.di.trans.steps.textfileinput.TextFileInputMeta

Branch Change:  + .53%

org.pentaho.di.trans.steps.uniquerows.UniqueRowsMeta

Instruction Change: -.18%:small_red_triangle_down:

org.pentaho.di.trans.Trans

Branch Change:  + .10%
Complexity Change:  + .14%
Instruction Change:  + .09%
Line Change:  + .11%

Checkstyle violations
The code submitted is not complying with one or more of our guidelines. To perform these checks locally, please use this tool.
/home/buildguy/project/engine/src/org/pentaho/di/trans/steps/filterrows/FilterRowsMeta.java

:heavy_exclamation_mark: Line has trailing spaces at Line: 455

:heavy_exclamation_mark: Line has trailing spaces at Line: 466

@e-cuellar very minor (annoying) checkstyle issues, let me know when they are cleaned :)

Build Completed
:fire: This pull request has some issues. It would be preferable to fix them in order for it to be just perfect. See below for more details. Some links are also available below for further assistance in addressing those issues.
Build Commands
ant -Dtestreports.dir=bin/reports/unit-test -f engine/build.xml clean-all resolve jacoco && ant -Dtestreports.dir=bin/reports/integration-test -f engine/build.xml jacoco-integration checkstyle publish-local

Cleanup Commands
rm -r ~/.ivy2/local || echo no publish local to remove

Changed files
  engine/src/org/pentaho/di/trans/steps/filterrows/FilterRowsMeta.java

Unit test coverage change
These statistics help you identify how your changes have affected the coverage of the following files. If a file is not in this list, then its coverage was not affected by your changes. To get some help interpreting these metrics, please refer to Jacoco's documentation.
org.pentaho.di.job.entries.evaluatetablecontent.JobEntryEvalTableContent

Branch Change:  + 2.02%
Complexity Change:  + 2.13%
Instruction Change:  + .34%

org.pentaho.di.job.entries.sql.JobEntrySQL

Instruction Change: -3.28%:small_red_triangle_down:
Line Change: -3.87%:small_red_triangle_down:

org.pentaho.di.job.entries.waitforsql.JobEntryWaitForSQL

Branch Change: -.66%:small_red_triangle_down:
Complexity Change: -.93%:small_red_triangle_down:
Instruction Change: -.13%:small_red_triangle_down:

org.pentaho.di.trans.steps.checksum.CheckSumMeta

Instruction Change: -.11%:small_red_triangle_down:

org.pentaho.di.trans.steps.dbproc.DBProcMeta

Instruction Change:  + .11%

org.pentaho.di.trans.steps.excelwriter.ExcelWriterStepMeta

Branch Change:  + 1.28%
Complexity Change:  + .78%
Instruction Change:  + .08%

org.pentaho.di.trans.steps.filterrows.FilterRowsMeta

Branch Change:  + 1.56%
Complexity Change:  + 1.67%
Instruction Change:  + 1.46%
Line Change:  + 1.25%

org.pentaho.di.trans.steps.groupby.GroupByMeta

Branch Change:  + 1.04%
Complexity Change:  + 1.05%
Instruction Change:  + .14%
Line Change:  + .37%

org.pentaho.di.trans.steps.ifnull.IfNullMeta

Branch Change:  + 2.00%
Complexity Change:  + 1.89%
Instruction Change:  + .17%

org.pentaho.di.trans.steps.insertupdate.InsertUpdateMeta

Instruction Change:  + .05%

org.pentaho.di.trans.steps.memgroupby.MemoryGroupByMeta

Branch Change:  + 1.18%
Complexity Change:  + 1.35%
Instruction Change:  + .18%
Line Change:  + .48%

org.pentaho.di.trans.steps.replacestring.ReplaceStringMeta

Instruction Change: -.07%:small_red_triangle_down:

org.pentaho.di.trans.steps.rowgenerator.RowGeneratorMeta

Instruction Change:  + .08%

org.pentaho.di.trans.steps.sort.SortRowsMeta

Instruction Change:  + .08%

org.pentaho.di.trans.steps.sortedmerge.SortedMergeMeta

Instruction Change: -.18%:small_red_triangle_down:

org.pentaho.di.trans.steps.synchronizeaftermerge.SynchronizeAfterMergeMeta

Instruction Change:  + .04%

org.pentaho.di.trans.steps.tableoutput.TableOutputMeta

Branch Change: -.91%:small_red_triangle_down:
Complexity Change: -.82%:small_red_triangle_down:


[mlt] Blacklist entries in Arabic script?

Maltese lexicon has one entry in Arabic script. I suspect it's ended there from an etymology section somewhere. Is there a way to blacklist scripts during scraping?

If you add an inner "scripts" dictionary with "ara" and latn entries, it'll have that effect.

The issue is that Maltese should not have entries in Arabic AFAIK.

It'll have that effect. Or if you want to be totally explicit you can say:
"scripts": { "latn": "Latn" }
We also do something similar, IIRC, to filter out a handful of non-Han entries under Mandarin.

This also drops all the spellings containing the apostrophe (').

Oh interesting. This is the ASCII one (0x27)?
If so let's make another bug to track that down. I would have thought
generalized_split.py would pass through punctuation (but not space or
dash), but if not it should be easy to add in.
On Sun, May 17, 2020 at 12:46 PM Alexander Gutkin<EMAIL_ADDRESS>wrote:

This also drops all the spellings containing the apostrophe (').
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/kylebgorman/wikipron/issues/166#issuecomment-629826882,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AABG4OI3NDXZ2KK5DFPWINTRSAIHJANCNFSM4NC27OMQ
.

Opened #172.


Improved state selection (CORE-23)

Correct the IES predicates used to answer states for an entity in response to a states query within the Telicent Graph Schema.

How fixed is this? Does it change with IES versions? Just wondering how we can be head of any future changes - or should the Data team be telling us?

Ideally you identify all flavours of state / temporal part by knowing the whole subProperty hierarchy of ies:isPartOf


Median age instead of mean age

Use median age instead of the mean age in key figures.

seems to be using median now


using configmap and grpc for configuration handling

Feature Request
Short Description
Kubearmor is provided with the configuration at invoke time. This means changing the default configuration is not possible without updating the container image.
Most of the configuration options are currently using default values but some may need to be changed depending upon deployment basis.
Similarly in systemd mode, the only way for the user to change the configuration is to edit /opt/kubearmor/kubearmor.yaml file.
Is your feature request related to a problem? Please describe the use case.
A lot of times it is required that the defaultPosture of Kubearmor be changed between Audit/Deny. Currently, it is not possible to change it in the context of k8s deployment unless the user changes the deployment yamls manually.
Describe the solution you'd like
Use grpc endpoint to save/update the configuration setting.
Use k8s configmap for saving the configuration. Observe changes to configmap and invoke the corresponding grpc handler.
For e.g., watchConfigMap() -> updateDefaultPosture() -> grpcUpdateDefaultPosture() -> updateDefaultPosture()

Hi @nyrahul, I'm using these configurations to change the defaultPosture to audit, but I'm still receiving events with defaultPosture policy and block action.
My configuration:
    spec:
      containers:
      - args:
        - -defaultFilePosture=audit
        - -hostDefaultFilePosture=audit

Pod's logs:
2022-08-11 19:18:47.523563	INFO	Configuration [{Cluster:default Host:xxxxx GRPC:32767 LogPath:/tmp/kubearmor.log SELinuxProfileDir:/tmp/kubearmor.selinux CRISocket: Visibility:process,file,network,capabilities HostVisibility:default Policy:true HostPolicy:false KVMAgent:false K8sEnv:true DefaultFilePosture:audit DefaultNetworkPosture:block DefaultCapabilitiesPosture:block HostDefaultFilePosture:audit HostDefaultNetworkPosture:block HostDefaultCapabilitiesPosture:block CoverageTest:false}]
2022-08-11 19:18:47.523585	INFO	Final Configuration [{Cluster:default Host:xxxxxxxxxxx GRPC:32767 LogPath:/tmp/kubearmor.log SELinuxProfileDir:/tmp/kubearmor.selinux CRISocket: Visibility:process,file,network,capabilities HostVisibility:none Policy:true HostPolicy:false KVMAgent:false K8sEnv:true DefaultFilePosture:audit DefaultNetworkPosture:block DefaultCapabilitiesPosture:block HostDefaultFilePosture:audit HostDefaultNetworkPosture:block HostDefaultCapabilitiesPosture:block CoverageTest:false}

Event:
== Alert / 2022-08-11 19:31:47.544754 ==
ClusterName: default
HostName: XXXXX
NamespaceName: XXXX
PodName: XXXXX
Labels: XXXXX
ContainerName: XXXX
ContainerID: XXXXX
ContainerImage: XXXXX
Type: MatchedPolicy
PolicyName: DefaultPosture
Source: /usr/local/bin/python
Resource: /usr/local/lib/python/XXXXXXXX
Operation: File
Action: Block
Data: syscall=SYS_OPENAT fd=-100 flags=O_WRONLY|O_CREAT|O_EXCL|O_CLOEXEC
Enforcer: AppArmor
Result: Permission denied
HostPID: XXX
HostPPID: XXX
PID: XXX
PPID: XXX
ParentProcessName: /usr/local/bin/python
ProcessName: /usr/local/bin/python
UID: XXXX

Using that configuration shouldn't just audit these events?

Hey there!
I'd like to work on this issue

Hey there! I'd like to work on this issue

Thank you for the interest. Any questions, we are on slack.

@nyrahul  I would like to work on this

Lets consider a scenario,

Current global config default posture for file, network and capability is audit.
There is a namespace temp and it's annotated with kubearmor-network-posture=block.
therefore default posture for namespace temp will be:

{
  defaultFilePosture:         audit,
  defaultCapabilitiesPosture: audit,
  defaultNetworkPosture:      block,
}

Let global default posture is changed to block now.
Now defaultPosture for namespace temp can be
either

Considered namespace temp to be annotated completely and using existing posture without any change

{
  defaultFilePosture:         audit,
  defaultCapabilitiesPosture: audit,
  defaultNetworkPosture:      block,
}

or

If we consider namespace posture/annotations to some extent and changed posture which are not defined yet with namespace annotations.

{
	defaultFilePosture:         block,    # changed because previously not defined
	defaultCapabilitiesPosture: block,    # changed because previously not defined
	defaultNetworkPosture:      block,
}

What should be the preferred case?
I've built the solution locally considering the case 1. In first look, i don't think it will add too much complexity to go with  case 2. but we need to discuss the applicability first.

I think it makes sense to consider the case 2.

WIP at my fork, trying to sync up with @s1ntaxe770r will push on top of existing PR #832
I've refactored the entire logic, added tests, ignore over-commenting/logging :grin:
PTAL  @daemon1024 , @nyrahul


Create mytopic.md

@briantest237 : Thanks for your contribution! The author(s) have been notified to review your proposed change.

@briantest237 : Thanks for your contribution! The author(s) have been notified to review your proposed change.

#sign-off

#sign-off

#sign-off


Fixed some warnings.

Codecov Report

Merging #6 into master will increase coverage by 0.04%.
The diff coverage is 0%.

@@            Coverage Diff             @@
##           master       #6      +/-   ##
==========================================
+ Coverage   21.21%   21.26%   +0.04%
==========================================
  Files          12       12
  Lines        2819     2808      -11
  Branches      356      356
==========================================
- Hits          598      597       -1
+ Misses       2183     2173      -10
  Partials       38       38

Impacted Files
Coverage Δ

source/broadband/webpa_adapter.c
24.23% <ø> (-0.26%)
:arrow_down:

source/broadband/webpa_notification.c
0% <ø> (ø)
:arrow_up:

source/broadband/webpa_internal.c
21.62% <0%> (ø)
:arrow_up:

source/broadband/webpa_parameter.c
20.14% <0%> (+0.24%)
:arrow_up:

source/broadband/webpa_table.c
0% <0%> (ø)
:arrow_up:

source/broadband/webpa_attribute.c
0% <0%> (ø)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update b8cbce3...ef63183. Read the comment docs.


Add more types

@yinxusen , just  a couple minor comments otherwise looks good.  How is the string conversion coming, think you will have something tomorrow?

So i tried an implementation that might be correct, but my knowledge of the arrow java api is limited.
https://github.com/mariusvniekerk/spark-arrow/blob/master/src/main/scala/org/apache/spark/sql/arrow/DataframeToArrow.scala
You guys might be able to steal ideas from there?

@BryanCutler Sorry I messed up the PR because I merged with spark master.
@mariusvniekerk Thanks for the reference, it is really helpful and more comprehensive.
I'll close this one and open a new PR.


Simple Memory Game

PR Description 📜
I have added a new game called "Simple Memory Game".
Fixes #3960

Mark the task you have completed ✅

[x] I follow CONTRIBUTING GUIDELINE & CODE OF CONDUCT of this project.
[x] I have performed a self-review of my own code or work.
[x] My changes generates no new warnings.
[x] I have followed proper naming convention showed in CONTRIBUTING GUIDELINE
[x] I have added screenshot for website preview in assets/images
[x] I have added entries for my game in main README.md
[x] I have added README.md in my folder
[x] I have added working video of the game in README.md (optional)
[x] I have specified the respective issue number for which I have requested the new game.

Add your screenshots(Optional) 📸

Thank you soo much for contributing to our repository 💗

i have uploaded the image and the video in the readme file please do check

please check

Thank you  @aursh24 , for your valuable time and contribution in our GameZone 💗.
It’s our GameZone, so Let’s build this GameZone altogether !!🤝
Hoping to see you soon with another PR again 😇
Wishing you all the best for your journey into Open Source🚀


Allow using tokenizer's default chat template or pass custom jinja chat template

Closes #1689
Summary of changes:

Adds tokenizer_default as option for chat_template in chat_template prompt strategy that allows using the chat template from tokenizer's config.json
Allows falling back to chat templates available in axolotl if tokenizer does not have a chat template
Allows passing in a custom chat template using a new option chat_template_jinja which is only considered when chat_template option is set to jinja

Why?
Many popular models are not trained with chatml format. As a result for the model to correctly learn chatml we have to turn on train_on_inputs which requires more compute and time. If we can use the model's already learned chat template we can just learn the output tokens

On a separate note, I would also like to explore how can we support passing a jinja template entirely via the config itself which might me more flexible

Thanks! Would you have some time to put together some tests for this?

I wonder if it is a good idea to use the chat_template available from the trained model's tokenizer_config.json as the default template?

Added a few tests

I wonder if it is a good idea to use the chat_template available from the trained model's tokenizer_config.json as the default template?

Yes, that is what this PR aims to do, if we all agree, we can change the default in config to tokenizer_default_fallback_chatml :)

Yes. I am sorry I just read the PR in more detail and found that's exactly what I was hoping for (what a coincidence!) so I removed my previous dumb question. Thanks for the contribution!

I have one question. Can this option be used together with the sharegpt conversation format?

I have one question. Can this option be used together with the sharegpt conversation format?

Unfortunately no, they are different templating libraries.

Thank you for working on this! I was needing just this...and was going to either integrate my custom chat template into axolotl (limits me on experimenting) or work on getting this feature going. Glad you beat me to it. Doesn't look like it'll conflict much with the changes I made for https://github.com/axolotl-ai-cloud/axolotl/pull/1756.

Thank you for working on this! I was needing just this...and was going to either integrate my custom chat template into axolotl (limits me on experimenting) or work on getting this feature going. Glad you beat me to it. Doesn't look like it'll conflict much with the changes I made for https://github.com/axolotl-ai-cloud/axolotl/pull/1756.

Amazing, would love to see both getting merged

Added chat_template_jinja option to provide a Jinja template as per @fozziethebeat's suggestion

Bumping for a review here @winglian

Regarding point 4, it seems that the jinja template from that repo is very different when compared to how fastchat/axolotl currently formats the prompt, and mistrals templates
FastChat/Axolotl currently places the system message with a single newline inside the first user turn
Mistral 0.1 places the system message with 2 newlines inside the first user turn, and with different spacing between [INST] tokens
the Mistral v3 tokenizer places the system message with 2 newlines inside the last user turn
The one in the linked repo puts the system prompt before the first [INST], I'm unsure if this is an oversight by the creator of that template/repo or deliberately chosen to be different from the official mistral template

I can get rid of the mistral template now, anyway this PR now allows passing custom jinja template

Bumping this again @winglian

Have addressed comments and updated docs to the best of my knowledge. Will test DPO/ORPO.
One thing I am not clear on - what should be the behavior is some one provides different chat templates across different datasets? It causes ambiguity in selecting which template to finally save in the tokenizer. Ideally, no one should do it this way but there is no validation to protect against this.

Hi @chiragjn ,
I'd like to point out a potential issue that might occur when training DPO with prompt strategy chat_template.default and using the base model's (or custom) chat template.
In many open source models (e.g. Phi-3-medium-128k-instruct), the chat template validates that the first message has the user role, and raises an exception otherwise.
Consider the code here:
        result["chosen"] = tokenizer.apply_chat_template(
            [chosen],
            add_generation_prompt=False,
            chat_template=chat_template_string,
            tokenize=False,
        )

We pass here [chosen], which is a list containing a single message with assistant role. Hence the validation mentioned above will cause raising an exception.
This can be solved by adding a dummy user message, e.g.:
        dummy_user_message = {"role": "user", "content": "dummy"}
        result["chosen"] = tokenizer.apply_chat_template(
            [dummy_user_message, chosen],
            add_generation_prompt=False,
            chat_template=chat_template_str,
            tokenize=False,
        )

For trimming the dummy message, no additional code is needed, as it will be trimmed by the lines that follow:
        chosen_strip_index = result["chosen"].find(chosen["content"])
        result["chosen"] = result["chosen"][chosen_strip_index:].rstrip()

These lines remove what comes before the actual message content (typically an assistant header / special token).
The same workaround is needed in the rejected case as well (here).

I notice also that the loss masking may have some issues for chat template (likely not due to this PR, but an old issue).
For ex, it sometimes does not detect the assistant portion correctly.
# user masked correctly
(-100, 235322) |(-100, 235371) im(-100, 571) _(-100, 235298) start(-100, 2997) |>(-100, 73786) user(-100, 1645)
(-100, 108) Each(-100, 7639)  problem(-100, 3210)  consists(-100, 13569)  of(-100, 576)  three(-100, 2149)
...
# assistant also masked
 <(-100, 235322) |(-100, 235371) im(-100, 571) _(-100, 235298) start(-100, 2997) |>(-100, 73786) assistant(-100, 105776)
(-100, 108) If(-100, 2495)  the(-100, 573)  third(-100, 4906)  statement(-100, 6218)  is(-100, 603)  true(-100, 1382)  ((-100, 591) Bananas(-100, 217061)  cost(-100, 2458)

Config to recreate:
base_model: mlx-community/gemma-2-2b

datasets:
  - path: LDJnr/Puffin
    type: chat_template
    chat_template: chatml
    shards: 10
chat_template: chatml

Sorry, I did not mean to block this PR
It is just that I was running busy at my main job, I wanted to address the DPO issues before merging, I'll make some time for this

I'm excited to see this land!

Hey @chiragjn , I made some changes but did not have write permission to the PR. Could you take a look at the one I made to yours and merge it if it's okay?
I merged main to the branch, fixed some config, updated docs, and added some more tests.
@levguy , I added a test for the case you talked about, and it seemed like phi3's chat template did not have that issue https://github.com/truefoundry/axolotl/blob/dd87d8c438998b60dceaa5f5093ba619c3bca683/tests/prompt_strategies/test_dpo_chat_templates.py#L167-L190

Apologies, life just got too busy to get back to this.
@NanoCode012 thanks for carrying it 🙌 , I have forwarded all your commits. Not sure why it was not editable by maintainers in first place.
Changes seem good to me

Hey @chiragjn , I made some changes but did not have write permission to the PR. Could you take a look at the one I made to yours and merge it if it's okay?
I merged main to the branch, fixed some config, updated docs, and added some more tests.
@levguy , I added a test for the case you talked about, and it seemed like phi3's chat template did not have that issue https://github.com/truefoundry/axolotl/blob/dd87d8c438998b60dceaa5f5093ba619c3bca683/tests/prompt_strategies/test_dpo_chat_templates.py#L167-L190

Hey @NanoCode012 ,
Thank you for adding a test for that case!

Your test uses a conversation without a system message. You need to add system message for testing this.
The chat template of Phi-3 was updated by model owners and it supports system message now, so it can't be used for testing this issue. You can use google/gemma-2b-it instead. Here we see that its chat template raises an exception if system message is passed:

"chat_template": "{{ bos_token }}{% if messages[0]['role'] == 'system' %}{{ raise_exception('System role not supported') }}{% endif %}{% ...

levguy

Hey @NanoCode012 ,
Thank you for adding a test for that case!
The chat template of Phi-3 was updated by model owners and the relevant validity check was removed. You can use google/gemma-2b-it instead. You can take a look at its chat template here. There is this part:
{% if (message['role'] == 'user') != (loop.index0 % 2 == 0) %}{{ raise_exception('Conversation roles must alternate user/assistant/user/assistant/...') }}

Which will cause an exception raised in case the first message has assistant role.

Hey @chiragjn , I made some changes but did not have write permission to the PR. Could you take a look at the one I made to yours and merge it if it's okay?
I merged main to the branch, fixed some config, updated docs, and added some more tests.
@levguy , I added a test for the case you talked about, and it seemed like phi3's chat template did not have that issue https://github.com/truefoundry/axolotl/blob/dd87d8c438998b60dceaa5f5093ba619c3bca683/tests/prompt_strategies/test_dpo_chat_templates.py#L167-L190

Hey @NanoCode012 ,
Thank you for adding a test for that case!
The chat template of Phi-3 was updated by model owners and the relevant validity check was removed. You can use google/gemma-2b-it instead. You can take a look at its chat template here. There is this part:
{% if (message['role'] == 'user') != (loop.index0 % 2 == 0) %}{{ raise_exception('Conversation roles must alternate user/assistant/user/assistant/...') }}

Which will cause an exception raised in case the first message has assistant role.

Hey @levguy , thank you for clarifying this. I have added a commit to address this along with a test case https://github.com/truefoundry/axolotl/pull/8/commits/d101cfc125ad6b879209ef32eb5422f27539c9b5
@chiragjn I made a PR to your branch. Could you please take a look and merge when you can? This should be the last changes required, but if you can enable contributors to have write access, that’ll make things more convenient.

Hey @levguy , thank you for clarifying this. I have added a commit to address this along with a test case truefoundry@d101cfc
@chiragjn I made a PR to your branch. Could you please take a look and merge when you can? This should be the last changes required, but if you can enable contributors to have write access, that’ll make things more convenient.

Hey @NanoCode012 , great, thank you!
As a final remark, maybe it would be better to change (in dummy_user_message) the "dummy" content to a very unlikely string such as "[[[dummy]]]" or something like this. This would reduce the chance of having a problem when calling find(chosen["content"]) and find(rejected["content"]) - a wrong behaviour will occur in case chosen["content"] and/or rejected["content"] are identical to the dummy string.

@chiragjn I made a PR to your branch. Could you please take a look and merge when you can? This should be the last changes required, but if you can enable contributors to have write access, that’ll make things more convenient.

I think I am not able to enable direct write access because I started the PR from my employer's fork 🤦
If it helps I can re-create this PR from my personal account fork.
Or NanoCode012, you can also close this PR and start a PR directly from your fork, I don't mind, either way commit history would be preserved

Superseded by #1970


[V2 metadata] Update device service with empty name should return status code 400

🐞 Bug Report
Affected Services

 The issue is located in: core-metadata
Description and Minimal Reproduction
Updating a device service without id and name properties returns status code 404 correctly, but updating a device service with empty name or empty id returns 400.
$curl -X PATCH http://localhost:48081/api/v2/deviceservice -d '[{"service": {"adminState": "LOCKED"}}]'

{"apiVersion":"v2","requestId":"","message":"device service json decoding failed","statusCode":400}

🔥 Exception or Error
Update a device service with empty name and without id returns status code 404, but it should be 400.
$curl -X PATCH http://localhost:48081/api/v2/deviceservice -d '[{"service": {"name":"", "adminState": "LOCKED"}}]'

[{"apiVersion":"v2","requestId":"","message":"fail to query object *models.DeviceService, because : v2:deviceService:name doesn't exist in the database","statusCode":404}]

Update a device service with empty id and without name returns status code 404, but it should be 400.
$curl -X PATCH http://localhost:48081/api/v2/deviceservice -d '[{"service": {"id":"", "adminState": "LOCKED"}}]'

[{"apiVersion":"v2","requestId":"","message":"fail to query object *models.DeviceService, because id: v2:deviceService: doesn't exist in the database","statusCode":404}]

🌍  Your Environment
EdgeX Version:
master

This is not a bug.  When other APIs are called to update something (Intervalue, ValueDescriptor, etc.) we return 400 when the request is "bad" because no id or name is provided.  It does look like the error code is consistent across APIs.


No canvas for websites

For these set of urls [ acer.com, drive.google.com,icloud.com,onedrive.live.com,mega.nz ] no canvas is rendered .

acer.com redirect to http://us.acer.com/ac/en/US/content/home
drive.google.com redirect to login page

then the problem is the redirects, to work have to use canonical urls.

Okay


tensor flow does not support operator.truediv

tensorflow type does not support operation.truediv as in the basic example in https://tensorflow.googlesource.com/tensorflow/+/master/tensorflow/g3doc/tutorials/word2vec/word2vec_basic.py
In line 153,  ' normalized_embeddings = embeddings / norm'  will cause the attribute error as followed
"AttributeError: type object 'Tensor' has no attribute 'truediv"
To solve the problem, we can replace this line as 'normalized_embeddings = tf.div(embeddings, norm)'

Fixed by 1d76583411038767f673a0c96174c80eaf9ff42f.  Also see #1 .

Thankyou for your help


Allow building an libpq adapter.Tx publicly similarly to other adapters

Mostly useful when wanting to ensure atomicity of operations on the client side with EnqueueTx.

Codecov Report

Merging #25 into master will decrease coverage by 1.74%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master      #25      +/-   ##
==========================================
- Coverage   89.84%   88.10%   -1.75%
==========================================
  Files           6        6
  Lines         266      227      -39
==========================================
- Hits          239      200      -39
  Misses         16       16
  Partials       11       11

Impacted Files
Coverage Δ

job.go
76.47% <0.00%> (-2.48%)
:arrow_down:

worker.go
83.01% <0.00%> (-1.47%)
:arrow_down:

client.go
98.24% <0.00%> (-0.20%)
:arrow_down:

client_option.go
100.00% <0.00%> (ø)

worker_option.go
100.00% <0.00%> (ø)

adapter/zap/logger.go
100.00% <0.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 117c502...5272733. Read the comment docs.

Thank you for contribution.
I agree that having consistent interface totally makes sense, but the library is already released and Tx struct is public, so making it private will break the contract and will require new major release. Pls leave struct as it is now to keep the contract, and the builder function can exist as well to make adapters interface more consistent.

Great! Thank you!


NPM

It seems the package is not available on NPM anymore

Is it really needed? npm install cferdinandi/smooth-scroll will save the dep as github:cferdinandi/smooth-scroll.

Ah that's right, I forgot the namespace ^^
But it's easier to have it directly on NPM ;)

On the other hand, I don't think my installation process (because it's integrated in a framework I'm using for my enterprise) will install it as expected. It's a big issue to me.

Never mind, it worked very well. Sorry and thanks for your help ;)

It actually was never available on NPM. I manage a few dozen OSS projects, so for the sake of sanity, I support package managers exclusively through Git hooks. Glad you got things figured out, and @lkraav, thanks for jumping in!

This has been a pain when running npm shrinkwrap. It would be nice to be able to pin at a specific version via our package.json. It's not that hard to push to npm when you update the version...

@mockdeep Totally get it. You actually already can pin a specific version directly from GitHub:
git://github.com/user/repo.git#v0.0.1

**sources: **

http://www.devthought.com/2012/02/17/npm-tricks/
http://stackoverflow.com/a/16350674
https://docs.npmjs.com/files/package.json#git-urls-as-dependencies

I get asked about this a lot (along with Bower and other popular package managers). Here's the thing: I actively maintain a few dozen open source projects. Every package manager, even the popular ones like NPM, introduce additional maintenance and overhead in my workflow on something that I spend a lot of time developing and give away for free.
The popular ones, including NPM, already support all of the important things directly from GitHub without me needing to register and host code there, so there's extra work on my end (across dozens of projects) for a minimal increase in value for the end users.
In other words, I hear you, but it's just not gonna happen.

Legit ^^

We decided to run with zenscroll instead. We were getting errors when trying to shrinkwrap this like:
npm ERR! invalid: have<EMAIL_ADDRESS>(expected: github:cferdinandi/smooth-scroll)

It's hard enough getting npm to behave consistently without throwing stuff like this in the mix, and definitely not worth the struggle for something as small as a smooth scrolling library.

:+1:


Explore using pytest-xdist to speed up CI jobs.

See https://pypi.org/project/pytest-xdist/ .
The github runner has 2 cores so we could possibly cut down the job time by half.
The  CI time is not a big issue atm, but it will become one after we start running tests like https://github.com/nexB/vulnerablecode/pull/365 .
The pytest version we use is also outdated, so we need to

Use latest pytest
Use the xdist plugin

Fixed via https://github.com/nexB/vulnerablecode/commit/1c201288be411eddfd3ae0a610377e13ad060e9f and https://github.com/nexB/vulnerablecode/commit/5300ce990e18d3c6836a7eb1ff7bed27baebecd7 .
There has been slight improvement in speed. Not sure whether the imporvement is masked by importers taking longer duration to run (eg redhat importer now has backoff).


added reference to Job and Build states to the amazon eventbride page

Added references to Build and Job states to the amazon eventbridge page
In the PR:

Thank you for this submission, @nithyaasworld!
I made a small suggestion.


CephSaltExecutor: check existing deployment: Use local_cmd

local().cmd() doesn't properly check for errors (e.g. result is a str)
Fixes #87
Signed-off-by: Sebastian Wagner<EMAIL_ADDRESS>

Looks like this PR is an alternative fix to #85

Better error-checking is always a good idea IMO

@sebastian-philipp please rebase

rebased


docs(faq): more information on what changed in Helm 3

builds upon the blog posts recently written about some of the changes in Helm 3. This also captures some major features that were identified in #5695.
collides with #5582 (cc @hickeyma)
closes #5397
Signed-off-by: Matthew Fisher<EMAIL_ADDRESS>

This PR was a helpful read. Thought I'd point these minor nits out which I found as I read through the whole diff.


HasRetvalPerm - strings are treated as Iterable and returned as List insted of String

Describe the Bug

In our project we are using the class HasRetvalPerm to ensure object level permissions to our types.
An example of how we define a query is:
@strawberry_django.type
class MyType:
    my_string_field: Optional[str] = strawberry.field(
        resolver=lambda: "my_string",
        extensions=[HasRetvalPerm(MY_TYPE_PERMISSION)]
    )

However, when I run for example the following query
query {
   myType {
      myStringField
   }
}

the field is not resolved and instead I get this error:
"String cannot represent value: ['m', 'y', '_', 's', 't', 'r', 'i', 'n', 'g']"
What I assumed is happening is that the value is transformed into a list before being returned.
After digging a bit into the implementation, I realised that in the _resolve_obj there is this check:
    def _resolve_obj(
        self,
        source: Any,
        user: UserType,
        obj: Any,
        *,
        info: Info,
    ) -> Any:
        context = perm_context.get()
        if context.is_safe:
            return obj

        if isinstance(obj, Iterable):
            return list(self._resolve_iterable_obj(source, user, obj, info=info))

Since isinstance(obj, Iterable) is True if obj is a string, the return value will be trasformed into a list.
Expected behaviour
A string should not be treated as Iterable in this case
System Information

Operating system:
Strawberry version (if applicable): 0.235.1
Strawberry Django version: 0.46.1

Additional Context

Upvote & Fund

We're using Polar.sh so you can upvote and help fund this issue.
We receive the funding once the issue is completed & confirmed by you.
Thank you in advance for helping prioritize & fund our backlog.

Hi @rickypag ,
Hrm, the main issue here is that HasRetvalPerm will check permissions against the "retval", which is expected to be a django model object.
Is there a reason why you are trying to use HasRetvalPerm in a field that returns a string? If that field is in a DjangoType parent, you could use HasSourcePermission instead.
Will leave this issue open to understand more of your use case and understand if this is really an issue or not.

Hi @bellini666,
Thank you for pointing this out. I was indeed using the wrong class permission and with HasSourcePerm I managed to make the type work as expected!

@rickypag awesome, so I'm going to close this issue. Let me know if you find something else


DocsArchive-<IP_ADDRESS>Auto archive content

Please don't merge this PR until target archive repo PR https://github.com/OPS-E2E-PPE/docs-archive-test-target_ed2acab1-d3c0-4392-884c-635ca1cc0fef/pull/1 is merged into live branch.
Auto archive content to https://github.com/OPS-E2E-PPE/docs-archive-test-target_ed2acab1-d3c0-4392-884c-635ca1cc0fef.git

Docs Build status updates of commit bdc2803:
:x: Validation status: errors
Please follow instructions here which may help to resolve issue.

File
Status
Preview URL
Details

:x:Error

Details

[Error]

Cannot sync git repo to specified commit because branch Release_Archive_master_2020-02-22-07-05-33 has been deleted or has been force pushed: fatal: couldn't find remote ref refs/heads/Release_Archive_master_2020-02-22-07-05-33

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.


Improvement for k8s.io/docs/setup/independent/install-kubeadm/

Give a command that I can execute to validate the installation was successful.

please provide more details about your OS/distribution version and what setup step you wish to verify.

/language en
/triage information-needed

Hi @adobo76 , simply running kubeadm command shows a help page which signifies a successful installation. Not sure If you wish to verify just kubeadm or other packages too.

There needs to be some sort of self test that validates the install.
On Wed, Jun 12, 2019 at 12:05 PM Prankul<EMAIL_ADDRESS>wrote:

Hi @adobo76 https://github.com/adobo76 , simply running kubeadm command
shows a help page which signifies a successful installation. Not sure If
you wish to verify just kubeadm or other packages too.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/kubernetes/website/issues/14828?email_source=notifications&email_token=AHTS3ZKTCAZT2J6IXXCGUJDP2E3IPA5CNFSM4HXARXGKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODXRJYIY#issuecomment-501390371,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AHTS3ZPQIVWV3DM2ITUXVVDP2E3IPANCNFSM4HXARXGA
.

the install validation is already part of the DEBs and RPMs on the package managers side.
if the package failed to install tools like apt-get and rpm will fail.
/close

That only works in the case of when you use a package manager though.
In this case I followed the linux container tab.
On Wed, Jun 12, 2019 at 1:25 PM Lubomir I. Ivanov<EMAIL_ADDRESS>wrote:

the install validation is already part of the DEBs and RPMs on the package
managers side.
if the package failed to install tools like apt-get and rpm will fail.
/close
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/kubernetes/website/issues/14828?email_source=notifications&email_token=AHTS3ZIOTA4Q4VZFYINPJG3P2FETJA5CNFSM4HXARXGKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODXRRM6I#issuecomment-501421689,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AHTS3ZNADG2H7CH3WKDHW5TP2FETJANCNFSM4HXARXGA
.

xref https://kubernetes.io/docs/setup/production-environment/tools/kubeadm/install-kubeadm/#installing-kubeadm-kubelet-and-kubectl
well that's why i asked about the OS/distro.
curl -L --remote-name-all https://storage.googleapis.com/kubernetes-release/release/${RELEASE}/bin/linux/amd64/{kubeadm,kubelet,kubectl}
curl -L --remote-name-all https://storage.googleapis.com/kubernetes-release/release/${RELEASE}/bin/linux/amd64/{kubeadm,kubelet,kubectl}.md5

md5sum -c <<< "$(cat kubeadm.md5) kubeadm"
md5sum -c <<< "$(cat kubelet.md5) kubelet"
md5sum -c <<< "$(cat kubectl.md5) kubectl"

a PR to add this verification is welcome.
/sig cluster-lifecycle

I did manage to get beyond these steps yesterday.
Certainly checking the md5sums is a good first step, however, having a
network addon seems to be a requirement, there are several ports that need
to be available and I remember it complaining about my docker version as
well.
Doesn't kubernetes require docker to run?  A check for that would be a good
step as well.
Aaron
On Wed, Jun 12, 2019 at 3:20 PM Lubomir I. Ivanov<EMAIL_ADDRESS>wrote:

xref
https://kubernetes.io/docs/setup/production-environment/tools/kubeadm/install-kubeadm/#installing-kubeadm-kubelet-and-kubectl
well that's why i asked about the OS/distro.
curl -L --remote-name-all https://storage.googleapis.com/kubernetes-release/release/${RELEASE}/bin/linux/amd64/{kubeadm,kubelet,kubectl}
curl -L --remote-name-all https://storage.googleapis.com/kubernetes-release/release/${RELEASE}/bin/linux/amd64/{kubeadm,kubelet,kubectl}.md5
md5sum -c <<< "$(cat kubeadm.md5) kubeadm"
md5sum -c <<< "$(cat kubelet.md5) kubelet"
md5sum -c <<< "$(cat kubectl.md5) kubectl"
a PR to add this verification is welcome.
/sig cluster-lifecycle
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/kubernetes/website/issues/14828?email_source=notifications&email_token=AHTS3ZI5R4VHQIOTDIX4HK3P2FSBXA5CNFSM4HXARXGKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODXR2VMQ#issuecomment-501459634,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AHTS3ZIZZFQ66TCZOXIBCMDP2FSBXANCNFSM4HXARXGA
.

however, having a network addon seems to be a requirement, there are several ports that need
to be available and I remember it complaining about my docker version as
well. Doesn't kubernetes require docker to run?  A check for that would be a good
step as well.

installing a container runtime (CR) is right before installing the kubeadm binaries in the above link.
kubeadm will perform network and CR checks on init and join runtime (they are called "preflight checks") to verify if your nodes are k8s ready.
this is the kubeadm "create a cluster" guide:
https://kubernetes.io/docs/setup/production-environment/tools/kubeadm/create-cluster-kubeadm/
the #kubeadm channel on the k8s slack should be a good starting point if want help with your setup.
https://slack.k8s.io/


GEODE-10068: Make WanCopyRegionFunctionService thread pool configurab…

…le through property

For all changes:

[ ] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message?

[ ] Has your PR been rebased against the latest commit within the target branch (typically develop)?

[ ] Is your initial contribution a single, squashed commit?

[ ] Does gradlew build run cleanly?

[ ] Have you written or updated unit tests to verify your changes?

[ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?

@albertogpz you don't need my approval... if you need to get this completed, move it out of the draft status and have the Code Owners review it.

@albertogpz you don't need my approval... if you need to get this completed, move it out of the draft status and have the Code Owners review it.

@kohlmu-pivotal thanks for telling me. I had forgotten that it was still in draft status.

@albertogpz you don't need my approval... if you need to get this completed, move it out of the draft status and have the Code Owners review it.

@kohlmu-pivotal thanks for telling me. I had forgotten that it was still in draft status.

@kohlmu-pivotal It seems that I need now also your approval because you requested changes.


question about what compressor does for me...

Hi
I might have the wrong impression of what autoCompressor does for me.
I assume like everybody else here, I am using MVC to serve up different pages. Each of these pages is pretty much its own SPA with a fair bit of typescript /javascript code spread across many files.
From my initial reading of this project, I was assuming that autoCompressor would analyse each page's javascript and build a bundle per MVC page. The end result would be that each MVC page did one round trip to the server to pull all the needed javascript in a single minimized file. And that this minimized file would contain just the javascript for that MVC page.
Is that the case? If so, I don't understand the autoBundler configuration to achieve that.
I have my typescript files organized this way:
Scripts/
-> Api // files that provide me access to my MVC Web Api back end (models and functions that do ajax calls)
-> App // MVVM view model classes broken out per page, but there could still be some sharing here
-> Controllers // the initial kick off typescript file per Controller Action - this is new since adopting this project
-> Libs // 3rd party libraries I am using.
I guess I am expecting to get to a point where I could have a common include for the app (say jquery, bootstrap and knockout or angular), and then for the controller Actions: Home/Action1 and Home/Action2, I would have two bundles automatically created based on the smallest subset of my typescript code, one called Home/Action1Bundle.js and the other called Home/Action2Bundle.js.
Am I thinking about this correctly?
Thanks a lot
Greg

Yeah, thats how Ive been working towards as well for the last week. Unfortunately Ive been pretty sick too, but my important takeaway is that:

YuiCompressor doesn't handle exports.default, so you must name all exports in your scripts.
Things crash if you compress jquery, so you need to exclude that from your bundles and reference the precompressed file.
autocompress path system is really broken. Ive spent some time rewriting it with proper tests but it is still not stable...
/Mattias
Den 19 nov. 2016 08:14 skrev gregveres notifications@github.com:Hi
I might have the wrong impression of what autoCompressor does for me.
I assume like everybody else here, I am using MVC to serve up different pages. Each of these pages is pretty much its own SPA with a fair bit of typescript /javascript code spread across many files.
From my initial reading of this project, I was assuming that autoCompressor would analyse each page's javascript and build a bundle per MVC page. The end result would be that each MVC page did one round trip to the server to pull all the needed javascript in a single minimized file. And that this minimized file would contain just the javascript for that MVC page.
Is that the case? If so, I don't understand the autoBundler configuration to achieve that.
I have my typescript files organized this way:
Scripts/
-> Api // files that provide me access to my MVC Web Api back end (models and functions that do ajax calls)
-> App // MVVM view model classes broken out per page, but there could still be some sharing here
-> Controllers // the initial kick off typescript file per Controller Action - this is new since adopting this project
-> Libs // 3rd party libraries I am using.
I guess I am expecting to get to a point where I could have a common include for the app (say jquery, bootstrap and knockout or angular), and then for the controller Actions: Home/Action1 and Home/Action2, I would have two bundles automatically created based on the smallest subset of my typescript code, one called Home/Action1Bundle.js and the other called Home/Action2Bundle.js.
Am I thinking about this correctly?
Thanks a lot
Greg

—You are receiving this because you are subscribed to this thread.Reply to this email directly, view it on GitHub, or mute the thread.

On Nov 19, 2016, at 2:16 AM, Mattias Åslund<EMAIL_ADDRESS>wrote:
Yeah, thats how Ive been working towards as well for the last week. Unfortunately Ive been pretty sick too, but my important takeaway is that:

Great. I am not alone. :) Thanks for confirming. Right now I am in the process of migrating to requireJS. Once that is done, I will turn my attention to the bundling.

YuiCompressor doesn't handle exports.default, so you must name all exports in your scripts.

Ok. That shouldn’t be a problem. I am only using the import X = require(‘x’) format for 3rd party libraries.

Things crash if you compress jquery, so you need to exclude that from your bundles and reference the precompressed file.

Ah that’s good to know. I think I managed to get the compressor to run once and I had jQuery in a bundle but I was getting an error. I thought it very strange that it would error out on jQuery.
I will leave out jQuery then.
Right now I am using MVC’s bundling for jQuery, bootstrap and knockout because I need them on every page.

autocompress path system is really broken. Ive spent some time rewriting it with proper tests but it is still not stable...
/Mattias

Hi @aquamoth
I think we may be pushing in the wrong direction. Now that I have a clearer understanding of the tool and I understand the terms better, I have re-read the bundling page. The example they show is assuming that there is a single bundle per "Area", not per "Action", which is a huge difference.
I need to keep reading, but for right now it doesn't even look possible to automate creating a bundle per page (action). I think I would have to add a new config file entry for every page I create. This is the kind of maintenance I was hoping to avoid.
I guess I could go back and start breaking down my app into fairly small "Areas".

Hi @aquamoth
Were you running into errors like this:
RequireJsNet.Compressor.2.2.5\build\RequireJsNet.Compressor.targets(33,5): error : Could not load script C:\Users\gregv\Source\Repos\SquashSpider\SquashSpider\SquashSpider\Scripts\App\FrontDeskFrontDesk.js
Notice the last part of the path is FrontDeskFrontDesk.js. There should be a slash between FrontDesk and FrontDesk.js.
I believe it is picking this up from a require statement that the Typescript compiler created:
define(["require", "exports", 'App/FrontDesk/FrontDesk'], function (require, exports, FrontDesk_1) {

Yeah, the autobundling’s entire path system is totally broken. Im rewriting it but had too much sick kids the last two weeks to finish it.
When I do I will push my branch and make a pull-request. Im not sure they take it though since it is some major rewriting including unit-testing, but then there is always my branch to use….
/Mattias
Från: gregveres<EMAIL_ADDRESS>Skickat: den 25 november 2016 09:26
Till: vtfuture/RequireJSDotNet<EMAIL_ADDRESS>Kopia: Mattias Åslund<EMAIL_ADDRESS>Mention<EMAIL_ADDRESS>Ämne: Re: [vtfuture/RequireJSDotNet] question about what compressor does for me... (#82)
Hi @aquamoth https://github.com/aquamoth
Were you running into errors like this:
RequireJsNet.Compressor.2.2.5\build\RequireJsNet.Compressor.targets(33,5): error : Could not load script C:\Users\gregv\Source\Repos\SquashSpider\SquashSpider\SquashSpider\Scripts\App\FrontDeskFrontDesk.js
Notice the last part of the path is FrontDeskFrontDesk.js. There should be a slash between FrontDesk and FrontDesk.js.
I believe it is picking this up from a require statement that the Typescript compiler created:
define(["require", "exports", 'App/FrontDesk/FrontDesk'], function (require, exports, FrontDesk_1) {
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub https://github.com/vtfuture/RequireJSDotNet/issues/82#issuecomment-262868543 , or mute the thread https://github.com/notifications/unsubscribe-auth/AFNKNxJNg5-XdmzT5eOjmjQfkvd_GbFaks5rBkc0gaJpZM4K3FOH .  https://github.com/notifications/beacon/AFNKN0PnnBD6VnZIWkqeA-Q46x6njz1iks5rBkc0gaJpZM4K3FOH.gif

Im not sure we are trying for the exact same level of bundling.
I want a global bundle, an area bundle and possibly some page-specific scripts, that I may or may not bundle depending on how many they are.
I assume I will just make an autobundle entry for each bundle I need, but I don’t think it would be hard to provide a directory of entrypoints and tell it to make a bundle for each entrypoint, if that is what you really need/want.
I AM a bit worried there will be more code to download to clients with the latter setup though, since each page will load its own bundle with a lot of overlapping scripts. But your mileage may vary.. :)
/Mattias
Från: gregveres<EMAIL_ADDRESS>Skickat: den 25 november 2016 09:18
Till: vtfuture/RequireJSDotNet<EMAIL_ADDRESS>Kopia: Mattias Åslund<EMAIL_ADDRESS>Mention<EMAIL_ADDRESS>Ämne: Re: [vtfuture/RequireJSDotNet] question about what compressor does for me... (#82)
Hi @aquamoth https://github.com/aquamoth
I think we may be pushing in the wrong direction. Now that I have a clearer understanding of the tool and I understand the terms better, I have re-read the bundling page http://requirejsnet.veritech.io/compressor.html . The example they show is assuming that there is a single bundle per "Area", not per "Action", which is a huge difference.
I need to keep reading, but for right now it doesn't even look possible to automate creating a bundle per page (action). I think I would have to add a new config file entry for every page I create. This is the kind of maintenance I was hoping to avoid.
I guess I could go back and start breaking down my app into fairly small "Areas".
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub https://github.com/vtfuture/RequireJSDotNet/issues/82#issuecomment-262867844 , or mute the thread https://github.com/notifications/unsubscribe-auth/AFNKN1HGYocI-pzUHikZuEURj3u1ZQyOks5rBkVRgaJpZM4K3FOH .  https://github.com/notifications/beacon/AFNKNy-pyxMe1zbPwj3h_ES0yWLQ3WYLks5rBkVRgaJpZM4K3FOH.gif

@aquamoth thank you for carrying on the conversation with me. I am trying to learn from others so you sharing how you are doing things is very helpful to me. I appreciate it.
I wouldn't mind hearing from @rangp. He has created a fork of this project to fix the compressor as well. I have to assume that he fixed the bugs in the path system, but he says he is using the MVC bundling system to ensure cache busting.
Ok I see what you are saying. I think my reason for bundling is to minimize the JS code (through uglify) and then to minimize the number of round trips to the server.
I could see that by bundling too high up in the hierarchy you end up delivering too much code to the browser if the user isn't fully exploring that area of the product. And to avoid that you might create very fine grained Areas within your source tree.
So your classification of bundles comes down to:

Global (things that every page needs: jQuery, knockout, bootstrap, in my case)

Area scripts (what controls the contents of this bundle? are you hand tuning this?)

page scripts (the left overs after your area bundling is done?)

I was thinking about the problem a little differently. I believe the shared scripts on a page are due to shared components being used on multiple pages. If I am correct, then I would like to have the ability to create a bundle from the support scripts for a component. I am using Typescript and single responsibility principle so one component sometimes ends up spanning multiple files. I would like to create a bundle for each component. Then there is always going to be the page's view model script that is specific to that page.
So I was looking for a solution that does the following:

global bundle

component bundles

page bundle

I don't want to load any unbundled scripts. It is the minimization of the script that is just as important to me as the joining of the minimized script with other scripts.
Right now I am using MVC's bundling to bundle my global scripts outside of requireJS and RequireJSNet. I would prefer to pull them into the same structure but that's only because I don't like exceptions. :)


Allow JSONField to be filtered as well

Inclusive of unit tests
closed https://github.com/AltSchool/dynamic-rest/issues/296

Should take a look at this other PR https://github.com/AltSchool/dynamic-rest/pull/250


add scripts to upload test cases

Add scripts to /upload-test-cases to facilitate extracting test-cases from the feedback app's MongoDB collection, and opening a pull request with the new additions in pelias/acceptance-tests.

@dianashk , @hkrishna , once you check this over I'll get it onto the feedback app's server.

+1


xSQLServerAlwaysOnService: isHadrEnabled null in Get-TargetResource

I'm running into an issue where xSQLServerAlwaysOnService fails even though it is correctly setting the isHadrEnabled setting to True.  After it sets the value and restarts, it calls Test-TargetResource again, which first calls Get-TargetResource.  In Get-TargetResource, the verbose message throws an "Index operation failed; the array index evaluated to null." when it tries to show the status of Hadr.  Interestingly, it falls back through to Test-TargetResource which then is able to see the same value as True.  I have no idea why this is happening, but it kills the DSC at this point and I can't continue with the AlwaysOn setup.  Any help would be appreciated.  My DSC log output is below.  This is version <IP_ADDRESS> of xSQLServer on Windows 2012 R2 and SQL Server 2014 SP2.
VERBOSE: [2017-04-19T22:07:22] [VERBOSE] [myserver]: LCM:  [ Start  Resource ]
[[xSQLServerAlwaysOnService]AlwaysOnService::[AlwaysOnInstall]InstallSQL]
VERBOSE: [2017-04-19T22:07:22] [VERBOSE] [myserver]: LCM:  [ Start  Test     ]
[[xSQLServerAlwaysOnService]AlwaysOnService::[AlwaysOnInstall]InstallSQL]
VERBOSE: [2017-04-19T22:07:22] [VERBOSE] [myserver]:
[[xSQLServerAlwaysOnService]AlwaysOnService::[AlwaysOnInstall]InstallSQL] 2017-04-19_22-07-22: Connected to SQL
myserver\myinstance
VERBOSE: [2017-04-19T22:07:23] [VERBOSE] [myserver]:
[[xSQLServerAlwaysOnService]AlwaysOnService::[AlwaysOnInstall]InstallSQL] 2017-04-19_22-07-23: SQL Always On is disabled on
"myserver\myinstance".
VERBOSE: [2017-04-19T22:07:23] [VERBOSE] [myserver]:
[[xSQLServerAlwaysOnService]AlwaysOnService::[AlwaysOnInstall]InstallSQL] 2017-04-19_22-07-23: SQL Always On is in the desired
state for "myserver\myinstance": False.
VERBOSE: [2017-04-19T22:07:23] [VERBOSE] [myserver]: LCM:  [ End    Test     ]
[[xSQLServerAlwaysOnService]AlwaysOnService::[AlwaysOnInstall]InstallSQL]  in 0.0620 seconds.
VERBOSE: [2017-04-19T22:07:23] [VERBOSE] [myserver]: LCM:  [ Start  Set      ]
[[xSQLServerAlwaysOnService]AlwaysOnService::[AlwaysOnInstall]InstallSQL]
VERBOSE: [2017-04-19T22:07:23] [VERBOSE] [myserver]:
[[xSQLServerAlwaysOnService]AlwaysOnService::[AlwaysOnInstall]InstallSQL] 2017-04-19_22-07-23: Importing SQLPS module.
VERBOSE: [2017-04-19T22:07:27] [VERBOSE] [myserver]:
[[xSQLServerAlwaysOnService]AlwaysOnService::[AlwaysOnInstall]InstallSQL] 2017-04-19_22-07-27: Enabling Always On for the
instance myserver\myinstance
VERBOSE: [2017-04-19T22:07:28] [VERBOSE] [myserver]:
[[xSQLServerAlwaysOnService]AlwaysOnService::[AlwaysOnInstall]InstallSQL] 2017-04-19_22-07-28: SQL Always On has been enabled
on "myserver\myinstance". Restarting the service.
VERBOSE: [2017-04-19T22:07:28] [VERBOSE] [myserver]:
[[xSQLServerAlwaysOnService]AlwaysOnService::[AlwaysOnInstall]InstallSQL] 2017-04-19_22-07-28: Connected to SQL
myserver\myinstance
VERBOSE: [2017-04-19T22:07:28] [VERBOSE] [myserver]:
[[xSQLServerAlwaysOnService]AlwaysOnService::[AlwaysOnInstall]InstallSQL] 2017-04-19_22-07-28: Getting SQL Service information
VERBOSE: [2017-04-19T22:07:28] [VERBOSE] [myserver]:
[[xSQLServerAlwaysOnService]AlwaysOnService::[AlwaysOnInstall]InstallSQL] 2017-04-19_22-07-28: SQL Server service restarting
VERBOSE: [2017-04-19T22:07:35] [VERBOSE] [myserver]:
[[xSQLServerAlwaysOnService]AlwaysOnService::[AlwaysOnInstall]InstallSQL] 2017-04-19_22-07-35: Starting SQL Server Agent (myinstance)
VERBOSE: [2017-04-19T22:07:35] [VERBOSE] [myserver]:
[[xSQLServerAlwaysOnService]AlwaysOnService::[AlwaysOnInstall]InstallSQL] 2017-04-19_22-07-35: Connected to SQL
myserver\myinstance
VERBOSE: [2017-04-19T22:07:38] **[ERROR] Index operation failed; the array index evaluated to null.**
VERBOSE: [2017-04-19T22:07:39] [VERBOSE] [myserver]:
[[xSQLServerAlwaysOnService]AlwaysOnService::[AlwaysOnInstall]InstallSQL] 2017-04-19_22-07-39: SQL Always On is in the desired
state for "myserver\myinstance": True.
VERBOSE: [2017-04-19T22:07:39] [VERBOSE] [myserver]: LCM:  [ End    Set      ]
[[xSQLServerAlwaysOnService]AlwaysOnService::[AlwaysOnInstall]InstallSQL]  in 16.0790 seconds.
VERBOSE: [2017-04-19T22:07:39] [ERROR] The PowerShell DSC resource
'[xSQLServerAlwaysOnService]AlwaysOnService::[AlwaysOnInstall]InstallSQL' with SourceInfo 'C:\Program
Files\WindowsPowerShell\Modules\AlwaysOnCompositeDSC\DSCResources\AlwaysOnInstall\AlwaysOnInstall.schema.psm1::786::13::xSQLSer
verAlwaysOnService' threw one or more non-terminating errors while running the Set-TargetResource functionality. These errors
are logged to the ETW channel called Microsoft-Windows-DSC/Operational. Refer to this channel for more details.
VERBOSE: [2017-04-19T22:07:39] [VERBOSE] [myserver]: LCM:  [ End    Set      ]
VERBOSE: [2017-04-19T22:07:39] [ERROR] The SendConfigurationApply function did not succeed.

It must be this line 41 that generates the error...
New-VerboseMessage -Message ( 'SQL Always On is {0} on "{1}\{2}".' -f @{$false='disabled'; $true='enabled'}[$sql.IsHadrEnabled],$SQLServer,$SQLInstanceName )

Could you please test and run this. Change to the correct host name and instance.
$null = [System.Reflection.Assembly]::LoadWithPartialName('Microsoft.SqlServer.Smo')
$sql = New-Object Microsoft.SqlServer.Management.Smo.Server localhost\SQL2016
@{$false='disabled'; $true='enabled'}[$sql.IsHadrEnabled]

Does it return enabled?

Yes, it returns 'enabled'.  Is it a timing thing coming so soon after the restart?

If you run the configuration again does it work then as well?
Only timing issue I can think of is if the state can be anything else than $false and $true, then there would be a index problem.
Wonder if we gonna rewrite that part of the code just in case, to a normal if()-block.

Yes, it works on rerun.  But it is not easily rerun in Prod by my operations team, as it has been packaged as a composite resource called by another DSC, and the credentials are passed to it.  So I need it to work consistently. :)

Nah, you should not have to re-run it :) Just wanted to make sure it wasn't a problem when it was run from the resource thru LCM.
I will make the code change as I mentioned above and we can see if that removes the problem. I will try to do that tomorrow. I thinking we also should add an error message if the state is neither $false or $true so we can catch if there is a timing issue. Then we can see how we handle it. Sounds okay?

Sounds good - thanks!

@iyoumans I try to write regression tests for this, and I can only reproduce this error when Server.IsHadrEnabled equals $null.
I will make the proposed change and we take it from there.

Thanks again

@iyoumans If you like you can test the code in PR #524. You find it in my working branch here https://github.com/johlju/xSQLServer/tree/chnages-to-xSQLServerAlwaysOnService.
I'm mostly interested if you can get it to throw an error. That is if the property IshadrEnabled is ever something else than $true or $false.