Skip absence-data on indexing

Relevant content transferred from http://dev.gbif.org/issues/browse/POR-2864, which was created 02/Oct/15
Description of problem
"Our current index is not able to interpret absence-data. For the time being, a published record to us means occurrence, ergo presence, not absence" - AH
How are users of GBIF.org affected?
"At the 8th EU Nodes workshop/meeting in Libson 18-21 April 2016 several participants reported that they noticed absence data been shown on GBIF.org." - KB
How are data publishers affected?
In the context of publishing sample event data, I instructed them to advise publishers in their network not to publish absence data through GBIF.org until we can handle it properly" - KB
What does absence-data look like?
Publishers may be publishing occurrence records with absence-data in the following fields/ways:
occurrenceStatus=absent
individualCount=0
organismQuantity=0

Instructions given to publishers on how to publish absences can be found in this IPT FAQ.
Hurdles and ideas for implementation

"we should probably already avoid creating a verbatim record? Or even not a fragment? Or interpretation removes the verbatim and/or fragment?" Alternatively we could keep all records incl absent ones, interpret the OccurrenceStatus from the different options listed above and then avoid them in cubes & maps and offer an OccurrenceStatus filter in solr. Maybe the cube should even just have another dimension present/absent.  - MD
"I would strongly prefer the second option, for 4 reasons:

Those records are useful (e.g. for modelling).
This is a representation issue. One could have them in the occurrence store, but remove them from maps (like how none georeferenced records are not shown), or as you suggest: create a filter/dimension for them and maybe shown them on maps in a different colour.
If absent records are removed during interpretation, it becomes much more difficult for the publisher to assess why the number of records in their source is different from the number of records presented on GBIF. That difference should indicate an issue in the process (e.g. records removed at the source are not deleted by GBIF, ...), not because valid absent records were ignored.
Absent records are going to become more ubiquitous, especially if GBIF can handle them nicely." - PD

"As for counts shown in the portal I think we should:

show all counts on http://www.gbif.org/
show all & georeferenced presence count on http://www.gbif.org/occurrence/
use presence only counts for all the metrics below the main search block on http://www.gbif.org/occurrence/
show all counts in dataset details
show detailed breakdwon by presence & absence on dataset stats page
include all records in a default search, but allow an occurrenceStatus enum based filter
occurrenceStatus enum currently has 7 values which we might want to reduce to just present, absent, doubtful: https://github.com/gbif/gbif-api/blob/master/src/main/java/org/gbif/api/vocabulary/OccurrenceStatus.java#L23" - MD

"Another issue is what do people do with the data when it's downloaded? Even if, say, GBIF hides absences on the maps, if that data is in the download then we are assuming that users are sufficiently savvy to handle absences correctly. if not, this could have dire consequences for naive efforts to model distributions (e.g., seabirds in the Alps)." - RP
"For downloads and similarly for API search/download calls for example via the R package we should simply default to exclude absence data, i.e. set an occurrenceStatus filter by default to include only presence. This way someone will only see absence data when they actively request it by changing the filter." - MD

My view is that we need to be really careful about this issue.  Fields may end up containing "0" as the value for many reasons, not all of them because of absences - it may just represent a missing value in a database.
That means that we should probably flag occurrence records with an issue if they have 0 in these positions, but not treat them for now at least as absences. Maybe we should carry out a sampling of records with 0 in these fields and see whether they seem to represent presences or absences.
For sampling-event data, it is messier.  We should place the emphasis on the fact that a sample event without a record for a species recorded elsewhere in the same dataset will be considered to mean that the species was not found in the event.  We could/should have ways to represent sampling event datasets as sparse matrices and perhaps allow zeroes to be populated in these (as an option) during download.
I think that we should not encourage people to share special absence records except in cases where it is hard for them to clear these out from sampling event datasets.  They should not be part of our world in other cases.

This is my attempt at raising the issue of 'absences' (after feedback from Dmitry, Ben and
Presence-Absence Data
Lee Belbin 8/2/2017
To an ecologist, ‘presence-absence data’ implies an absence of a species at a location. Presence-only data identifies that a species was present at a location. The most important misconception about presence-absence data is that absences are explicitly recorded in the field. This is rare, but the ALA does have some. Absences are usually inferred in most systematic surveys.
A transect of pitfall traps for invertebrates or a suite of quarter hectare plots for trees are typical examples of systematic surveys. Pitfall traps are placed in the ground at regular intervals along a transect, left for a period and then species (often with numbers) that have fallen into the traps are recorded. With the tree survey, the presence or number of all tree species with a diameter at breast height above a threshold may be recorded. The key aspect of such surveys is specific or implied taxonomic scope – what type of species were in the scope of the survey.
The data from systematic surveys will usually take the form of “Site X, latitude, longitude, species A, species B, species C…”. Each site will also have metadata and possibly site environmental data. There is no mention of species that were not seen at the site.
Absences can however be inferred by combining sites with similar survey type and taxonomic scope. The Darwin Core terms that may help to evaluate if sites can be meaningfully combined would include scientificName, class, order, family, vernacularName, samplingProtocol, establishmentMeans, occurrenceStatus, eventID, parentEventID, recordedBy, basisOfRecord, eventDate, sampleSizeUnit, samplingEffort, fieldNotes and eventRemarks (see http://tdwg.github.io/dwc/terms/).
Once candidate sites are identified, they can be combined by generating a sites by species matrix (=co-occurrence matrix). If we add two more sites to our Site X above, say “Site Y, species B, species D, species E” and “Site Z, species A, species D, species F”, we can generate the following table (matrix)  where “1” represents a presence and “0” represents an absence-
             Species A    Species B    Species C    Species D    Species E    Species F

Site X             1                 1                 1                 0                  0                  0
Site Y             0                 1                 0                 1                  1                  1
Site Z             1                 0                 0                 1                  0                  1
The absences are now explicit, for example “Species D is absent from X”.  The great difficulty is however determining what sites can be meaningfully combined. Could survey sites for trees be combined if they were of different sized plots? Probably. Could pitfall survey sites be combined if they span 100 years? Depends. Surveys may use the same gear but target different species types or resolve taxa to different levels. Careful evaluation of available Darwin Core terms is needed before combining sites.
Recommendations

The use of Darwin Core Event terms must be strongly encouraged. The ALA has systematic survey data but records have not been fully documented to enable effective re-use. Some systematic survey records have been imported without samplingProtocol etc.
samplingProtocol is a verbatim term. Values need to be classified into a useful subset. TDWG has initiated work on controlled vocabularies (Paula Zermoglio).
samplingProtocol and other Darwin Core terms are not currently indexed in the ALA, so sites can only be combined if they are exported with the full Darwin Core terms, evaluated and then processed. But what sites to export? Catch-22.
A standard suite of core tests/assertions and all Darwin Core terms need to be indexed for filtering to be possible. GBIF and TDWG are developing a standard test suite (lead by me).
Searching and filtering logic needs to support AND and OR. For example, “display sites that have (samplingProtocol=”pitfall” OR samplingProtocol=”most net”) and eventDate >=”1963-03-08T14:07-0600"1990 AND eventDate<"1999-03-08T14:07-0600".
Create a new tool in the ALA’s Spatial Portal to generate a sites by species matrix from a set of filtered sites. This matrix can then be used in tools such as SDM where 'true absences' (rather than pseudo absences) can be utilized.

The current method of having the data provider filter out absence records in the IPT is untenable. This prevents other data aggregators that harvest from IPT (OBIS for instance) from being able to bring in the absence records and creates a disconnect between any measurement or facts and the occurrence IDs that are link to them. As an example, I just recently published a dataset that I aligned with Event Core and used Extended Measurement or Facts to make accessible information about coral and macroalgae percent cover in the Virgin Islands (https://www1.usgs.gov/obis-usa/ipt/manage/resource.do?r=noaa_coralreefmonitoring_lpipercentcover) if I filter out records in the occurrence file that have occurrenceStatus = absent then measurements in the extended measurements or facts file will not have a corresponding occurrence ID to link to.

I could place a filter on the extended measurements or facts file as well as the occurrence file but in my opinion, valuable information is being lost to the user this way.

Thank you @albenson-usgs for a concise summary of the issue and the OBIS perspectove. As you can see from the issue tags and as discussed at TDWG 2016, the problem is high on GBIF to-do list, and perhaps we can remove needs-clarification tag and find assignee for this. @ahahn-gbif @timrobertson100. This issue is a potential blocker for the sampling event mobilization campaign.

Looking at the mobilization priority 2 https://www.gbif.org/news/4fkukOiVWoEMiKSa2meQ08/prioritizing-content-mobilization-in-the-gbif-network, the blocking nature of the absence issue needs some attention soon. Related - the OBIS paper on dealing with MeasurementOrFact and the Humboldt core paper are both out since we left this thread.

Hi Dmitry
Thanks for the link to the mobilization priorities. 'Gaps' have been a key
interest to me for some years.
Regards Priority 2, how do you see absences as a blocking issue (in GBIF)?
We handle them ok for analysis, but probably not so well more generally.
For example, if a taxon is mapped that contains both presences and
absences, you won't get colour differentiation unless you are in the
Spatial Portal and use faceting. We need some form of warning.
Lee
[image: untitled]
Lee Belbin
Science Advisor
Atlas of Living Australia
National Research Collections Australia, National Collections & Marine
Infrastructure, CSIRO | Clunies Ross Street, Acton ACT 2601 |
GPO Box 1700, Canberra  ACT  2601
Phone: +61 0419 374 133 *|<EMAIL_ADDRESS><EMAIL_ADDRESS>|
www.ala.org.au | http://www.csiro.au/en/Research/Collections
On Thu, Aug 24, 2017 at 7:37 PM, dschigel<EMAIL_ADDRESS>wrote:

Looking at the mobilization priority 2 https://www.gbif.org/news/
4fkukOiVWoEMiKSa2meQ08/prioritizing-content-mobilization-in-the-gbif-
network, the blocking nature of the absence issue needs some attention
soon. Related - the OBIS paper on dealing with MeasurementOrFact and the
Humboldt core paper are both out since we left this thread.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/gbif/portal16/issues/308#issuecomment-324585865, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AGWRXlSOH1recCfGovRBTZ9CGrn-aPozks5sbURogaJpZM4MRFSO
.

Hi Lee, all,
Yes, I think absences issue is a blocker for sampling event mobilization, for the data publishers. Sampling event data are observation attempts, both successful (presence or abundance) and failed (absences) made within a 4-dimensional space: time, space, methods, and taxonomic target. It is a burden for a publisher, and a loss of value for the statistical analyses for the users, to ask removing the zeros in the publication process. What's worse, the current process actually reduces the original quality of the dataset, as it makes zeros (attempted, but failed observations) equal to empty cells (no data), not good.

Thanks Dmitry
We certainly need to handle 'absence' data in a systematic and transparent
way, and with the GBIF-ALA-iDigBio alignments, we need an agreed strategy.
My perspective is that we don't want to lose any data. Absences can be
valuable in many situations (e.g., modeling), but we need to make them
obvious. I'll discuss the issue here at the ALA.
Lee
[image: untitled]
Lee Belbin
Science Advisor
Atlas of Living Australia
National Research Collections Australia, National Collections & Marine
Infrastructure, CSIRO | Clunies Ross Street, Acton ACT 2601 |
GPO Box 1700, Canberra  ACT  2601
Phone: +61 0419 374 133 *|<EMAIL_ADDRESS><EMAIL_ADDRESS>|
www.ala.org.au | http://www.csiro.au/en/Research/Collections
On Wed, Aug 30, 2017 at 5:28 PM, dschigel<EMAIL_ADDRESS>wrote:

Hi Lee, all,
Yes, I think absences issue is a blocker for sampling event mobilization,
for the data publishers. Sampling event data are observation attempts, both
successful (presence or abundance) and failed (absences) made within a
4-dimensional space: time, space, methods, and taxonomic target. It is a
burden for a publisher, and a loss of value for the statistical analyses
for the users, to ask removing the zeros in the publication process. What's
worse, the current process actually reduces the original quality of the
dataset, as it makes zeros (attempted, but failed observations) equal to
empty cells (no data), not good.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/gbif/portal16/issues/308#issuecomment-325906765, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AGWRXtBI9Zp1ni_HKd2uLcH9vIhpWQFGks5sdQ8rgaJpZM4MRFSO
.

We need a whole new approach to incorporating sampling-event data (of which absences are a subset) into our mapping, API and data download structures. I can think of lots of visualisations that might be useful and that would show both relative abundance measures and absences.  We just need to make some progress.  Structuring the locality-event-taxon cube is the first step.

@dschigel - I don't really believe that presentation of absences is a specially hard problem for us - provided we can avoid confusion around 0 or null being supplied as the abundance measure.  Unspecified values easily become absences.  In the context of sampling-event data, the key thing is the construction of the corresponding locality-event-taxon cube.

Happy to hear that this won't be too difficult to tackle. I have already published several datasets with absences so you can work against those if you need examples:
https://www.gbif.org/dataset/7606f22a-2121-4853-b3ab-2e693c6790dd
https://www.gbif.org/dataset/2501caa2-803b-4f6b-9e36-f9f5c72c0666
https://www.gbif.org/dataset/40bfac0e-a1b5-4c9e-862e-64327195f210
https://www.gbif.org/dataset/c684b43a-0147-459b-ae97-0e3d9757fb92

I wanted to add to this thread that @dschigel and I spoke at the TDWG 2017 annual meeting about this topic. He informed me that registering absence data in GBIF should not be an issue and that it's just a matter of how to display it appropriately. It sounds like there may be an sampling event display in the works. It also sounds like GBIF is interested in knowing what outputs users of sampling event data might want. I don't have a good answer right now but I'll ask around.

Just wondering what, if anything, has happened with this since 2017.

@albenson-usgs: A few weeks ago I listed all the records in the ALA that had something in dwc:samplingProtocol and retrieved 390,853 records, 99% of which were from 'Garden bird surveys'. Considering that around 30% of the 86 million ALA records are likely from systematic bird surveys, we have a huge loss of data. The only advances at the ALA end are a) now all record terms are at least indexed and exposed and b) I've prodded our Data Manager to pursue the full bird survey data. Until we have systematic surveys that are properly documented in Darwin Core, generating implied (but true) absences is impossible. The ALA does currently have 1,234,578 records from wwc:occurrenceStatus="absent". How any of this relates to current work to align GBIF and the ALA (and Living Atlases, iDigBio etc), one lives in hope.

Closing an old issue — this was done in 2020.


Added a call to super init in _TaggableManager

This fixes an issue I ran into with django-taggit and Django 1.8 -- the Django admin list view for one of my models failed to render, with this error:
AttributeError: '_TaggableManager' object has no attribute 'name'

This error resulted from a call to _TaggableManager.__str__(), which now requires that the manager have a name attribute.  BaseManager.__init__() sets the name attribute, so I thought it appropriate to let that happen here.

Thanks for the pull request! Is it possible you could you add a regression test for this?

I didn't think it warranted a regression test, but I guess I can add one that just tries str(_TaggableManager())?

I didn't think it warranted a regression test, but I guess I can add one that just tries str(_TaggableManager())?

Yeah, I guess that should be fine. What part of the Django admin causes the Manager to get called with __str__?

Traceback (most recent call last):
  File "/home/vagrant/.venv/lib/python3.4/site-packages/django/core/handlers/base.py", line 164, in get_response
    response = response.render()
  File "/home/vagrant/.venv/lib/python3.4/site-packages/django/template/response.py", line 158, in render
    self.content = self.rendered_content
  File "/home/vagrant/.venv/lib/python3.4/site-packages/django/template/response.py", line 135, in rendered_content
    content = template.render(context, self._request)
  File "/home/vagrant/.venv/lib/python3.4/site-packages/django/template/backends/django.py", line 74, in render
    return self.template.render(context)
  File "/home/vagrant/.venv/lib/python3.4/site-packages/django/template/base.py", line 209, in render
    return self._render(context)
  File "/home/vagrant/.venv/lib/python3.4/site-packages/django/test/utils.py", line 96, in instrumented_test_render
    return self.nodelist.render(context)
  File "/home/vagrant/.venv/lib/python3.4/site-packages/django/template/base.py", line 903, in render
    bit = self.render_node(node, context)
  File "/home/vagrant/.venv/lib/python3.4/site-packages/django/template/debug.py", line 79, in render_node
    return node.render(context)
  File "/home/vagrant/.venv/lib/python3.4/site-packages/django/template/loader_tags.py", line 135, in render
    return compiled_parent._render(context)
  File "/home/vagrant/.venv/lib/python3.4/site-packages/django/test/utils.py", line 96, in instrumented_test_render
    return self.nodelist.render(context)
  File "/home/vagrant/.venv/lib/python3.4/site-packages/django/template/base.py", line 903, in render
    bit = self.render_node(node, context)
  File "/home/vagrant/.venv/lib/python3.4/site-packages/django/template/debug.py", line 79, in render_node
    return node.render(context)
  File "/home/vagrant/.venv/lib/python3.4/site-packages/django/template/loader_tags.py", line 135, in render
    return compiled_parent._render(context)
  File "/home/vagrant/.venv/lib/python3.4/site-packages/django/test/utils.py", line 96, in instrumented_test_render
    return self.nodelist.render(context)
  File "/home/vagrant/.venv/lib/python3.4/site-packages/django/template/base.py", line 903, in render
    bit = self.render_node(node, context)
  File "/home/vagrant/.venv/lib/python3.4/site-packages/django/template/debug.py", line 79, in render_node
    return node.render(context)
  File "/home/vagrant/.venv/lib/python3.4/site-packages/django/template/loader_tags.py", line 135, in render
    return compiled_parent._render(context)
  File "/home/vagrant/.venv/lib/python3.4/site-packages/django/test/utils.py", line 96, in instrumented_test_render
    return self.nodelist.render(context)
  File "/home/vagrant/.venv/lib/python3.4/site-packages/django/template/base.py", line 903, in render
    bit = self.render_node(node, context)
  File "/home/vagrant/.venv/lib/python3.4/site-packages/django/template/debug.py", line 79, in render_node
    return node.render(context)
  File "/home/vagrant/.venv/lib/python3.4/site-packages/django/template/loader_tags.py", line 65, in render
    result = block.nodelist.render(context)
  File "/home/vagrant/.venv/lib/python3.4/site-packages/django/template/base.py", line 903, in render
    bit = self.render_node(node, context)
  File "/home/vagrant/.venv/lib/python3.4/site-packages/django/template/debug.py", line 79, in render_node
    return node.render(context)
  File "/home/vagrant/.venv/lib/python3.4/site-packages/django/template/loader_tags.py", line 65, in render
    result = block.nodelist.render(context)
  File "/home/vagrant/.venv/lib/python3.4/site-packages/django/template/base.py", line 903, in render
    bit = self.render_node(node, context)
  File "/home/vagrant/.venv/lib/python3.4/site-packages/django/template/debug.py", line 79, in render_node
    return node.render(context)
  File "/home/vagrant/.venv/lib/python3.4/site-packages/django/template/base.py", line 1271, in render
    _dict = func(*resolved_args, **resolved_kwargs)
  File "/home/vagrant/.venv/lib/python3.4/site-packages/django/contrib/admin/templatetags/admin_list.py", line 320, in result_list
    'results': list(results(cl))}
  File "/home/vagrant/.venv/lib/python3.4/site-packages/django/contrib/admin/templatetags/admin_list.py", line 296, in results
    yield ResultList(None, items_for_result(cl, res, None))
  File "/home/vagrant/.venv/lib/python3.4/site-packages/django/contrib/admin/templatetags/admin_list.py", line 287, in __init__
    super(ResultList, self).__init__(*items)
  File "/home/vagrant/.venv/lib/python3.4/site-packages/django/contrib/admin/templatetags/admin_list.py", line 225, in items_for_result
    result_repr = display_for_field(value, f)
  File "/home/vagrant/.venv/lib/python3.4/site-packages/django/contrib/admin/utils.py", line 398, in display_for_field
    return smart_text(value)
  File "/home/vagrant/.venv/lib/python3.4/site-packages/django/utils/encoding.py", line 56, in smart_text
    return force_text(s, encoding, strings_only, errors)
  File "/home/vagrant/.venv/lib/python3.4/site-packages/django/utils/encoding.py", line 90, in force_text
    s = six.text_type(s)
  File "/home/vagrant/.venv/lib/python3.4/site-packages/django/db/models/manager.py", line 81, in __str__
    return '%s.%s.%s' % (app, model._meta.object_name, self.name)
AttributeError: '_TaggableManager' object has no attribute 'name'

Sorry, I meant like visually where does the TaggableManager get rendered in the admin?

Good question -- I just tracked it down, and apparently our admin class for that model is doing something it shouldn't -- tags is in the field list, and though we have a tags() method in that admin class, it isn't being called -- Django is just using the tags relation on the model instead (which is of course the manager instance).  That's embarrassing...

Apparently str() also fails for a completely different reason in Django 1.7 that I don't know how to fix, and I can't think of a reason why it would be particularly useful or necessary to call str() on a manager, so I'm going to close this PR.

All good, thanks for the PR though!


Composer update to 2.4-RC3

During composer update I get:
[RuntimeException]
  Could not load package zendframework/zendframework in http://packagist.org: [UnexpectedValueException] Could not parse version constraint ^1.0: Invalid version string "^1.0"

Dash is new composer feature available in master branch. In our company we use only last "stable" version (current alpha9). I think we should also define supported composer version in zf2. Currently we are not able to test RC or switch to 2.4 in future :(

Related commit: https://github.com/zendframework/zf2/commit/126f91cf2039a179a91368795bdff92d68b934ed

I was unaware that the ^ notation had not been released in a stable version of Composer; I'll update immediately; thanks for the report!

Thank @weierophinney for fast feedback and fix! Looks good for me :+1:

@snapshotpl I've tagged 2.4.0rc4 at this time with this change; please let me know if you have any further issues!

@weierophinney Unfortunately we still get issue. After research we have found that in file .composer/cache/repo/https---packagist.org/provider-zendframework$zendframework.json in rc3 ^ exists and fails composer update. Possible solution for that is remove affected tag (rc3) from packagist.

@weierophinney any chance to remove RC3?

Removing tags is not an option
On Mar 30, 2015 8:43 AM, "Witold Wasiczko"<EMAIL_ADDRESS>wrote:

@weierophinney https://github.com/weierophinney any chance to remove
RC3?
—
Reply to this email directly or view it on GitHub
https://github.com/zendframework/zf2/pull/7369#issuecomment-87581111.


Nuqta not working even in Romanized Scripts that support it

लकड़ा is being transliterated as lakaड़A even with itrans or wx.

IIUC, ड़ does not have any standard transliteration in either of those schemes. What do you think it should be transliterated to in itrans?

.Da as per this Wikipedia.
Also, as per ISO 15919, ṛa works better.
And, as per aksharamukha's website,  in hk which does not support nuqta, it can be customized as r3a.

Related issue: #43.
This example looks fine now for me, but सड़क doesn't; what's the difference?
-> https://lipyantar.glitch.me/?devanagari=लकड़ा सड़क
(code: https://glitch.com/edit/#!/lipyantar?path=script.js%3A114%3A27)
Screenshot in case the link stops working:

This might help: https://www.fontspace.com/unicode/analyzer#e=4KSy4KSV4KWc4KS-IOCkuOCkoeCkvOCklQ
Could it be related to #26?


Error Compiling lsp-sample

npm run compile

produces this log
0 info it worked if it ends with ok
1 verbose cli [ 'C:\Development\Libs\nodejs\node.exe',
1 verbose cli   'C:\Development\Libs\nodejs\node_modules\npm\bin\npm-cli.js',
1 verbose cli   'run',
1 verbose cli   'compile:server' ]
2 info using<EMAIL_ADDRESS>3 info using<EMAIL_ADDRESS>4 verbose run-script [ 'precompile:server', 'compile:server', 'postcompile:server' ]
5 info lifecycle lsp--sample@0.0.1~precompile:server<EMAIL_ADDRESS>6 info lifecycle lsp--sample@0.0.1~compile:server<EMAIL_ADDRESS>7 verbose lifecycle lsp--sample@0.0.1~compile:server: unsafe-perm in lifecycle true
8 verbose lifecycle lsp--sample@0.0.1~compile:server: PATH: C:\Development\Libs\nodejs\node_modules\npm\bin\node-gyp-bin;c:\Development\Workspace\VsCode\vscode-extension-samples\lsp-sample\node_modules.bin;c:\Development\Workspace\Python\uModules\env\Scripts;c:\Development\Bin;C:\Development\Utils\ConEmu\ConEmuPack.161206\ConEmu\Scripts;C:\Development\Utils\ConEmu\ConEmuPack.161206;C:\Development\Utils\ConEmu\ConEmuPack.161206\ConEmu;C:\Program Files\Docker\Docker\Resources\bin;C:\WINDOWS\system32;C:\WINDOWS;C:\WINDOWS\System32\Wbem;C:\WINDOWS\System32\WindowsPowerShell\v1.0;C:\Program Files\Microsoft SQL Server\130\Tools\Binn;C:\Program Files (x86)\Microsoft Emulator Manager\1.0;C:\Program Files\Git\cmd;C:\Program Files (x86)\Windows Kits\10\Windows Performance Toolkit;C:\Program Files\Microsoft\Web Platform Installer;C:\Program Files\dotnet;C:\Program Files (x86)\Microsoft SQL Server\110\DTS\Binn;C:\Program Files (x86)\Microsoft SQL Server\120\DTS\Binn;C:\Program Files (x86)\Microsoft SQL Server\130\DTS\Binn;C:\Program Files (x86)\GtkSharp\2.12\bin;C:\Development\Libs\Java\jre8\bin;C:\Development\Libs\NintendoDevInteface;C:\Program Files\Nintendo\HostBridge;C:\Development\Libs\nodejs;C:\Program Files (x86)\NVIDIA Corporation\PhysX\Common;C:\Users\paul\AppData\Local\Microsoft\WindowsApps;C:\Program Files (x86)\Microsoft VS Code\bin;C:\Program Files (x86)\Microsoft VS Code Insiders\bin;C:\Users\paul\AppData\Roaming\npm;;c:\Development\Libs;c:\Development\Libs\Java\jdk1.8\bin;C:\Program Files\Git\cmd;c:\Development\Tools\SQLite;C:\Program Files\Mercurial;C:\Games2\Emulators\Commodore\WinVice\WinVice-3.0.0;C:\Games2\Emulators\Atari\Stella\Stella-2.4.1;c:\Development\Tools\Pico\pico-8_0.1.4d_windows;c:\Development\Tools\Voxatron\voxatron_0.3.4_windows;C:\Games2\Emulators\Atari\z26\z26v3.02.01x;c:\Development\Utils\DoxyGen\1.8.11;c:\Development\Utils\SysInternalsSuite;c:\Development\Tools\6502\Assemblers\acme\acme-0.95;c:\Development\Tools\6502\Assemblers\dasm\dasm-2.20;c:\Development\Tools\6502\Assemblers\kickass\kickass-4.13;c:\Development\Tools\6502\Assemblers\turbo\64tass-1.51.992;;C:\Program Files\opencbm;C:\Games2\Emulators\Mame\Mame183;c:\Progra~1\7-ZIP;;c:\Development\Libs\Python\Python35;c:\Development\Libs\Python\Python35\Scripts;
9 verbose lifecycle lsp--sample@0.0.1~compile:server: CWD: c:\Development\Workspace\VsCode\vscode-extension-samples\lsp-sample
10 silly lifecycle lsp--sample@0.0.1~compile:server: Args: [ '/d /s /c',
10 silly lifecycle   'cd server && npm run installServer && cd .. && tsc -p server/tsconfig.json' ]
11 silly lifecycle lsp--sample@0.0.1~compile:server: Returned: code: 2  signal: null
12 info lifecycle lsp--sample@0.0.1~compile:server: Failed to exec compile:server script
13 verbose stack Error<EMAIL_ADDRESS>compile:server: cd server && npm run installServer && cd .. && tsc -p server/tsconfig.json
13 verbose stack Exit status 2
13 verbose stack     at EventEmitter. (C:\Development\Libs\nodejs\node_modules\npm\node_modules\npm-lifecycle\index.js:280:16)
13 verbose stack     at emitTwo (events.js:126:13)
13 verbose stack     at EventEmitter.emit (events.js:214:7)
13 verbose stack     at ChildProcess. (C:\Development\Libs\nodejs\node_modules\npm\node_modules\npm-lifecycle\lib\spawn.js:55:14)
13 verbose stack     at emitTwo (events.js:126:13)
13 verbose stack     at ChildProcess.emit (events.js:214:7)
13 verbose stack     at maybeClose (internal/child_process.js:925:16)
13 verbose stack     at Process.ChildProcess._handle.onexit (internal/child_process.js:209:5)
14 verbose pkgid<EMAIL_ADDRESS>15 verbose cwd c:\Development\Workspace\VsCode\vscode-extension-samples\lsp-sample
16 verbose Windows_NT 10.0.15063
17 verbose argv "C:\Development\Libs\nodejs\node.exe" "C:\Development\Libs\nodejs\node_modules\npm\bin\npm-cli.js" "run" "compile:server"
18 verbose node v8.9.1
19 verbose npm  v5.5.1
20 error code ELIFECYCLE
21 error errno 2
22 error<EMAIL_ADDRESS>compile:server: cd server && npm run installServer && cd .. && tsc -p server/tsconfig.json
22 error Exit status 2
23 error Failed at the<EMAIL_ADDRESS>compile:server script.
23 error This is probably not a problem with npm. There is likely additional logging output above.
24 verbose exit [ 2, true ]
also tried with node 7.9.0

I was able to get past this by removing node 8.9 and installing 7.9 - now I have a different error when trying to open the client. The task watch:client fails with the following:
0 info it worked if it ends with ok
1 verbose cli [ 'C:\Development\Libs\nodejs\node.exe',
1 verbose cli   'C:\Development\Libs\nodejs\node_modules\npm\bin\npm-cli.js',
1 verbose cli   'run',
1 verbose cli   'watch:client' ]
2 info using<EMAIL_ADDRESS>3 info using<EMAIL_ADDRESS>4 verbose config Skipping project config: C:\Users\paul/.npmrc. (matches userconfig)
5 verbose stack Error: ENOENT: no such file or directory, open 'C:\Users\paul\package.json'
6 verbose cwd C:\Users\paul
7 error Windows_NT 10.0.15063
8 error argv "C:\Development\Libs\nodejs\node.exe" "C:\Development\Libs\nodejs\node_modules\npm\bin\npm-cli.js" "run" "watch:client"
9 error node v7.9.0
10 error npm  v4.2.0
11 error path C:\Users\paul\package.json
12 error code ENOENT
13 error errno -4058
14 error syscall open
15 error enoent ENOENT: no such file or directory, open 'C:\Users\paul\package.json'
16 error enoent ENOENT: no such file or directory, open 'C:\Users\paul\package.json'
16 error enoent This is most likely not a problem with npm itself
16 error enoent and is related to npm not being able to find a file.
17 verbose exit [ -4058, true ]
So I "think" this has something to do with my environment setup on Windows, but cannot tell for sure. Why would it be looking for a package.json in my home directory?

Somehow I was able to get everything running again. I am a little disturbed because I am not exactly sure "how" I was able to get it running again. Needless to say, the changes I did make were

re-installed node 7.9
re-installed vscode 1.18.0 64-bit

short of that, I really wish I knew what fixed it
I will close this ticket

problem resolved . . . "somehow"


Trace info is not passed when ThreadPoolTaskExecutor.getThreadPoolExecutor() is used

Describe the bug
Spring Cloud Sleuth: 3.1.1
Underlying ThreadPoolExecutor from auto configured ThreadPoolTaskExecutor is not wrapped by sleuth, so when I use underlying executor then I lose initial trace id.
Sample
@Service
public class SomeService {

    @Autowired
    private ThreadPoolTaskExecutor taskExecutor;

    public void run() throws ExecutionException, InterruptedException {
        Future<?> helloWorld = taskExecutor.getThreadPoolExecutor().submit(() -> log.info("Hello world"));
        helloWorld.get();
    }

You can wrap the taskExecutor.getThreadPoolExecutor() result in a trace representation manually

@marcingrzejszczak Maybe LazyTraceThreadPoolTaskExecutor.getThreadPoolExecutor() could wrap executor service and return TraceableExecutorService?
https://gitter.im/spring-cloud/spring-cloud-sleuth?at=639f074b3daaa326ba8dc233

Care for a PR?

If you would like us to look at this issue, please provide the requested information. If the information is not provided within the next 7 days this issue will be closed.

I found that ThreadPoolTaskExecutor.getThreadPoolExecutor should to return instance of class ThreadPoolExecutor.
public ThreadPoolExecutor getThreadPoolExecutor() throws IllegalStateException

So idea to return TraceableExecutorService from LazyTraceThreadPoolTaskExecutor.getThreadPoolExecutor() didn't work, because TraceableExecutorService is not child of ThreadPoolExecutor.

Hmm so wrapping taskExecutor.getThreadPoolExecutor() in a trace representation won't work?

If you would like us to look at this issue, please provide the requested information. If the information is not provided within the next 7 days this issue will be closed.

Closing due to lack of requested feedback. If you would like us to look at this issue, please provide the requested information and we will re-open the issue.


FR: Add support for client certificates

Many Kubernetes servers (such as Azure AKS) use client certificates to connect (client-certificate-data and client-key-data fields). As far as I see right now this should not be supported as kubewall does not import these connections from the kubeconfig.
It would be interesting to add support for these connections so that they can be used with this type of servers.

Hello @ogarcia
Thanks for reaching out.
This feature is supported I have tested it on Google Cloud, but I don't have access to Azure.
Google Cloud supports token-based auth as well. Is it something that be used with Azure?
Also on my Local, I have a kind and orbstack cluster that contains.
kind: Config
preferences: {}
users:
  - name: kwok-kwok
    user:
      client-certificate: /pathTo/pki/admin.crt
      client-key: /pathTo/pki/admin.key
  - name: orbstack
    user:
      client-certificate-data: <REDACTED>
      client-key-data: <REDACTED>

Can you try once using Add Config > Certificate

I will try to test on Azure and other with other providers once more.

In Azure, the configuration is similar to the following:
apiVersion: v1
clusters:
- cluster:
    certificate-authority-data: <REDACTED>
    server: https://servername.westeurope.azmk8s.io:443
  name: servername
contexts:
- context:
    cluster: servername
    user: clusterUser_servername
  name: servername
kind: Config
preferences: {}
users:
- name: clusterUser_servername
  user:
    client-certificate-data: <REDACTED>
    client-key-data: <REDACTED>
    token: <REDACTED>

If I try to enter the data by hand in Add Config > Certificate I get this error:
Failure
serverIP or name or clientCertData or clientKeyData is empty

In my case both the certificate-authority-data value and the client-certificate-data and client-key-data values are a single single line string (in base64). When I enter it by hand I get exactly the same error using that string as I did previously decoding the base64 content to the certificate and key.

One thing you can do is to have a main docker tag containing the latest version of the main git branch (created with a GitHub action). This way when you upload a patch we can test it directly with that container. :wink:
I have compiled the git version and now it accepts the data manually (it still doesn't load them from kubeconfig), but when I can only pass the certificate and the key it gives this error:
W0922 08:50:02.707692       3 reflector.go:561] k8s.io/client-go@v0.31.1/tools/cache/reflector.go:243: failed to list *v1.Namespace: Get "https://servername.westeurope.azmk8s.io:443/api/v1/namespaces?limit=500&resourceVersion=0": tls: failed to verify certificate: x509: certificate signed by unknown authority
E0922 08:50:02.707859       3 reflector.go:158] "Unhandled Error" err="k8s.io/client-go@v0.31.1/tools/cache/reflector.go:243: Failed to watch *v1.Namespace: failed to list *v1.Namespace: Get \"https://servername.westeurope.azmk8s.io:443/api/v1/namespaces?limit=500&resourceVersion=0\": tls: failed to verify certificate: x509: certificate signed by unknown authority" logger="UnhandledError"
I understand that to avoid this it would have to have the possibility to configure also the certificate-authority-data so it can check the certificate with that authority.

I will expand on logs as well to know more.
If you get a chance, please try adding insecure-skip-tls-verify: true flag to a new config.
apiVersion: v1
clusters:
- cluster:
    certificate-authority-data: <REDACTED>
    server: https://servername.westeurope.azmk8s.io:443
    insecure-skip-tls-verify: true <---------------------  ADDED THIS LINE
  name: servername
contexts:
- context:
    cluster: servername
    user: clusterUser_servername
  name: servername
kind: Config
preferences: {}
users:
- name: clusterUser_servername
  user:
    client-certificate-data: <REDACTED>
    client-key-data: <REDACTED>
    token: <REDACTED>

I will try to see what I can do with main branch, also will expand on logs so that we know more.
If you get a chance please try one thing.
apiVersion: v1
clusters:
- cluster:
    insecure-skip-tls-verify: true <-------------- ADD THIS LINE
    # certificate-authority-data: <REDACTED> <-------------- COMMENT OR REMOVE THIS LINE
    server: https://servername.westeurope.azmk8s.io:443
  name: servername
contexts:
- context:
    cluster: servername
    user: clusterUser_servername
  name: servername
kind: Config
preferences: {}
users:
- name: clusterUser_servername
  user:
    client-certificate-data: <REDACTED>
    client-key-data: <REDACTED>
    token: <REDACTED>

Yes, making that change imports it and it works without problems. :tada:

Many thanks for the confirmation!! 💯
I'm closing the issue, feel free to open or create new as you saw something.
PS: Wait for the new release 🙂


Implementation of Chef External Secrets Provider

Problem Statement
Introducing Chef External Secrets Provider.
This will enable users to seamlessly integrate Chef-based secret management with Kubernetes through the existing External Secrets framework.
Related Issue
issue: https://github.com/external-secrets/external-secrets/issues/2905
Related work

documentation: docs/provider/chef.md
api: apis/externalsecrets/v1beta1/secretstore_chef_types.go
implementation: pkg/provider/chef

Checklist

[x] I have read the contribution guidelines
[x] All commits are signed with git commit --signoff
[x] My changes have reasonable test coverage
[x] All tests pass with make test
[x] I ensured my PR is ready for review with make reviewable

The missed unsigned commits have been signed by me and my teammates but DCO check now complaining about other commits . @Skarlso Could you help me in resolving DCO check issue ?

@Skarlso @sebagomez

Alternatively, take the diff, and create another PR and start off fresh if squashing is too complicated.

As suggested, we are closing this PR as we are unable to resolve unsigned commits , latest PR: https://github.com/external-secrets/external-secrets/pull/3127


refactor(autoware_auto_control_msgs): replace autoware_auto_control_msgs with autoware_control_msgs

PR Type

Refactor

Related Links

https://github.com/autowarefoundation/autoware.universe/issues/3674
https://github.com/autowarefoundation/autoware.universe/pull/3677
https://github.com/tier4/tier4_ad_api_adaptor/pull/106
https://github.com/autowarefoundation/autoware_launch/pull/353

Description
Replace all usages of

https://github.com/tier4/autoware_auto_msgs/tree/tier4/main/autoware_auto_control_msgs/msg
with
https://github.com/autowarefoundation/autoware_msgs/tree/main/autoware_control_msgs/msg

Review Procedure
Make sure it compiles with the rest of the Autoware once it is merged along with the Autoware Universe PR:

https://github.com/autowarefoundation/autoware.universe/pull/3677
https://github.com/tier4/tier4_ad_api_adaptor/pull/106
https://github.com/autowarefoundation/autoware_launch/pull/353

Remarks

Pre-Review Checklist for the PR Author
PR Author should check the checkboxes below when creating the PR.

[x] Code follows [coding guidelines][coding-guidelines]
[x] Assign PR to reviewer

Checklist for the PR Reviewer
Reviewers should check the checkboxes below before approval.

[ ] Commits are properly organized and messages are according to the guideline
[ ] Code follows [coding guidelines][coding-guidelines]
[ ] (Optional) Unit tests have been written for new behavior
[ ] PR title describes the changes

Post-Review Checklist for the PR Author
PR Author should check the checkboxes below before merging.

[ ] All open points are addressed and tracked via issues or tickets
[ ] Write [release notes][release-notes]

@tkimura4 you seem to be a maintainer of this package, could you look into this too?

I'm closing all of my message migration PRs.


[BUG] Occasional ManagedIdentity failures

Library name and version
Azure.Storage.Blobs" Version="12.20.0"
Describe the bug
When using Managed Identity credential and RBAC, the app running on Azure App Service has the correct credentials and network connections.
The application will succeed and get all data from blob storage, but somewhere in the middle when the application is running, Managed Identity fails and gets a bad token.
This happens intermittently, but the problem is not resolved until app is restarted
services.AddAzureClients(x =>
        {
            if (EnvironmentEx.IsOffline)
            {
                x.AddBlobServiceClient("UseDevelopmentStorage=true");
            }
            else
            {
                x.AddBlobServiceClient(new Uri(config["ApplicationConfiguration:BlobUrl"] ?? throw new Exception("BlobUrl is missing")));
                x.UseCredential(EnvironmentEx.IsDevelopment ? new AzureCliCredential() : new ManagedIdentityCredential());
            }
        });

ManagedIdentityCredential.GetToken invoked. Scopes: [ https://storage.azure.com//.default ] ParentRequestId: 7cb8a1ff-5041-46aa-a693-dcb04a13f0e1
False MSAL <IP_ADDRESS> MSAL.NetCore .NET 8.0.6 Microsoft Windows 10.0.20348 [2024-08-01 13:08:02Z - d196e687-f152-46b0-bee9-d4bbcd4454ae] MSAL MSAL.NetCore with assembly version '<IP_ADDRESS>'. CorrelationId(d196e687-f152-46b0-bee9-d4bbcd4454ae)
False MSAL <IP_ADDRESS> MSAL.NetCore .NET 8.0.6 Microsoft Windows 10.0.20348 [2024-08-01 13:08:02Z - d196e687-f152-46b0-bee9-d4bbcd4454ae] === AcquireTokenForClientParameters === SendX5C: False ForceRefresh: False
False MSAL <IP_ADDRESS> MSAL.NetCore .NET 8.0.6 Microsoft Windows 10.0.20348 [2024-08-01 13:08:02Z - d196e687-f152-46b0-bee9-d4bbcd4454ae] === Request Data === Authority Provided? - True Scopes - https://storage.azure.com//.default Extra Query Params Keys (space separated) - ApiId - AcquireTokenForClient IsConfidentialClient - True SendX5C - False LoginHint ? False IsBrokerConfigured - False HomeAccountId - False CorrelationId - d196e687-f152-46b0-bee9-d4bbcd4454ae UserAssertion set: False LongRunningOboCacheKey set: False Region configured:
False MSAL <IP_ADDRESS> MSAL.NetCore .NET 8.0.6 Microsoft Windows 10.0.20348 [2024-08-01 13:08:02Z - d196e687-f152-46b0-bee9-d4bbcd4454ae] === Token Acquisition (ClientCredentialRequest) started: Scopes: https://storage.azure.com//.default Authority Host: login.microsoftonline.com
False MSAL <IP_ADDRESS> MSAL.NetCore .NET 8.0.6 Microsoft Windows 10.0.20348 [2024-08-01 13:08:02Z - d196e687-f152-46b0-bee9-d4bbcd4454ae] [Region discovery] Not using a regional authority.
False MSAL <IP_ADDRESS> MSAL.NetCore .NET 8.0.6 Microsoft Windows 10.0.20348 [2024-08-01 13:08:02Z - d196e687-f152-46b0-bee9-d4bbcd4454ae] Skipping Instance discovery for Aad authority because it is not enabled.
False MSAL <IP_ADDRESS> MSAL.NetCore .NET 8.0.6 Microsoft Windows 10.0.20348 [2024-08-01 13:08:02Z - d196e687-f152-46b0-bee9-d4bbcd4454ae] Fetched access token from host login.microsoftonline.com.

ManagedIdentityCredential.GetToken invoked. Scopes: [ https://storage.azure.com//.default ] ParentRequestId: 8755a181-7153-469f-82a8-3e8d354f56f9
False MSAL <IP_ADDRESS> MSAL.NetCore .NET 8.0.6 Microsoft Windows 10.0.20348 [2024-08-01 13:14:32Z - 66d07183-a899-497d-8435-ea84f6ce7862] MSAL MSAL.NetCore with assembly version '<IP_ADDRESS>'. CorrelationId(66d07183-a899-497d-8435-ea84f6ce7862)
False MSAL <IP_ADDRESS> MSAL.NetCore .NET 8.0.6 Microsoft Windows 10.0.20348 [2024-08-01 13:14:32Z - 66d07183-a899-497d-8435-ea84f6ce7862] === AcquireTokenForClientParameters === SendX5C: False ForceRefresh: False
False MSAL <IP_ADDRESS> MSAL.NetCore .NET 8.0.6 Microsoft Windows 10.0.20348 [2024-08-01 13:14:32Z - 66d07183-a899-497d-8435-ea84f6ce7862] === Request Data === Authority Provided? - True Scopes - https://storage.azure.com//.default Extra Query Params Keys (space separated) - ApiId - AcquireTokenForClient IsConfidentialClient - True SendX5C - False LoginHint ? False IsBrokerConfigured - False HomeAccountId - False CorrelationId - 66d07183-a899-497d-8435-ea84f6ce7862 UserAssertion set: False LongRunningOboCacheKey set: False Region configured:
False MSAL <IP_ADDRESS> MSAL.NetCore .NET 8.0.6 Microsoft Windows 10.0.20348 [2024-08-01 13:14:32Z - 66d07183-a899-497d-8435-ea84f6ce7862] === Token Acquisition (ClientCredentialRequest) started: Scopes: https://storage.azure.com//.default Authority Host: login.microsoftonline.com
False MSAL <IP_ADDRESS> MSAL.NetCore .NET 8.0.6 Microsoft Windows 10.0.20348 [2024-08-01 13:14:32Z - 66d07183-a899-497d-8435-ea84f6ce7862] [Region discovery] Not using a regional authority.
False MSAL <IP_ADDRESS> MSAL.NetCore .NET 8.0.6 Microsoft Windows 10.0.20348 [2024-08-01 13:14:32Z - 66d07183-a899-497d-8435-ea84f6ce7862] [Instance Discovery] Skipping Instance discovery because it is disabled.
False MSAL <IP_ADDRESS> MSAL.NetCore .NET 8.0.6 Microsoft Windows 10.0.20348 [2024-08-01 13:14:32Z - 66d07183-a899-497d-8435-ea84f6ce7862] [ClientCredentialRequest] Acquiring a token from the token provider.
False MSAL <IP_ADDRESS> MSAL.NetCore .NET 8.0.6 Microsoft Windows 10.0.20348 [2024-08-01 13:14:32Z - 66d07183-a899-497d-8435-ea84f6ce7862] Checking client info returned from the server..
False MSAL <IP_ADDRESS> MSAL.NetCore .NET 8.0.6 Microsoft Windows 10.0.20348 [2024-08-01 13:14:32Z - 66d07183-a899-497d-8435-ea84f6ce7862] Saving token response to cache..

Expected behavior
Managed Identity credential consistently works on App Service
Actual behavior
Managed Identity disconnects and fails to make calls to blob gateway
Reproduction Steps
services.AddAzureClients(x =>
        {
            if (EnvironmentEx.IsOffline)
            {
                x.AddBlobServiceClient("UseDevelopmentStorage=true");
            }
            else
            {
                x.AddBlobServiceClient(new Uri(config["ApplicationConfiguration:BlobUrl"] ?? throw new Exception("BlobUrl is missing")));
                x.UseCredential(EnvironmentEx.IsDevelopment ? new AzureCliCredential() : new ManagedIdentityCredential());
            }
        });

Environment
No response

Upload file will still work, download file does not

CommitBlockListAsync works fine, blockBlobClient.DownloadToAsync(memoryStream) does not

Hi @garrettlondon1

but somewhere in the middle when the application is running, Managed Identity fails and gets a bad token.

What error or logging are you seeing that indicates that the managed identity is failing and that a bad token is received?

Hey @christothes , I apologize, I do not have any errors or logs indicating that managed identity is failing, and a bad token is received, specifically.
All resources from blob storage cannot be loaded, and when the App Service is restarted, everything works again instantly. I've reproduced it probably 5+ times.
I know for certain that the App service has access to the storage account because the same storage account hosts the zip file which powers the app service deployment via private endpoint
This is on an "always on" app service, blazor server monolith, using the absolute default managed identity setup

Does your service hang or does it continue to be responsive after the problem? Is the request to storage still occurring after the problem? If so, what is the response from the service?

The service completely hangs, although I will say on "12.21.1" version, have not encountered in 6+ days

That makes sense - I think the issue won't reproduce on 12.21.1 or later. This is due to a fix in one of its dependencies in Azure.Core that was shipped in version 1.41.0. The fix is here


Fix some spelling typos

Should be non-semantic.
Uses https://en.wikipedia.org/wiki/Wikipedia:Lists_of_common_misspellings/For_machines to find likely typos.

@bwignall thanks for submitting this, the fixes look good but seem to have caused some followup flake8 errors.

Thank you for highlighting that. I have added/squashed a commit that should handle that (a local flake8'ing showed the numba/tests/test_typedlist.py errors fixed and nothing in the other instances I checked).

@bwignall thanks for submitting the fixes, they do look good and seem to fix the CI errors with flake8 we saw previously!
Once small note: within the Numba project we tend to avoid using squash/rebase and force pushes after PRs have been reviewed so that we can track what happened during the review process. It isn't a big deal in this case, but please do keep that in mind should you decide to contribute more to Numba in future.

@esc , I am sorry for that. Some, but not all, projects prefer squashing, so I should not have assumed beforehand. I will try to keep this in mind in the future. Thank you!

@bwignall no problem, thanks for being considerate, we appreciate it.


DOC: add Numpy import  to scipy.sparse examples

Reference issue

#16759
What does this implement/fix?

Added import numpy as np to docstring examples in scipy.sparse
Additional information

The one failed CI job was not related to this change.  Thank again, @iwbc-mzk.


LLaMa-2-13B-chat deployed on azureml studio giving out of context response

Issue you'd like to raise.
Recently, we re-deployed the llam2-13-B chat model on azure ML Studio, whenever we are trying to do in-contex learning (RAG QA), always replied back with out of context response.
Currently we are passing request payload as using langchain:
System prompt: sys msg + context
Human prompt: question
Response: out of context response
2nd variation:
Human prompt: context + question
Response: junk answers like \n
Any suggestions who have experienced?
And, we are also followed the required input format.
Suggestion:
No response

Hi,
From my experience: make sure that your entire prompt follows the Meta's guidelines as to how it should be format. There is ready to go method that formats the list of messages in the unmerged PR that you can copy.
Second, System prompt should contain the general description of what the model is expected to do, like "you will be given text passages and question. Answer the question using information from the context". Follow that up with the HumanMessage that contains context and question, Passages: ... Question: ...
This should lead to better outputs


Not correctly parsing public suffix for blogspot.com

The tldjs package correctly parsed "subdomain.blogspot.com" as having a public suffix of "blogspot.com".
This tldts package seems to not recognize blogspot.com as a public suffix, as noted in the Public Suffix List.
Steps to reproduce:
const tldjs = require("tldjs");
const tldts = require("tldts");
const input = "subdomain.blogspot.com";
if (tldjs.getPublicSuffix(input) !== tldts.getPublicSuffix(input)) {
  throw new Error(`${input} did not match in both libraries`);
}

Hi @davidbielik and thank you for taking the time to open an issue,
This one is a bit tricky and is the expected behavior (although that might seem counter-intuitive). Let me try to explain. There are two sections in the public suffix list: ICANN domains and private domains. In tldjs both sections will be looked-up by default. At the time we found that this behavior was surprising to a lot of users of the library because it contains suffixes like blogspot.com or us-east-1.amazonaws.com which means that the domain of myapp.us-east-1.amazonaws.com is actually myapp.us-east-1.amazonaws.com whereas a lot of people would have expected amazonaws.com.
Long story short, depending on your use-case you might want to lookup the private section of the list or not and in tldts the default is to not do it (which is somewhat an opinionated choice). You can enable it using an option:
const { parse } = require('tldts');
parse('subdomain.blogspot.com', { allowPrivateDomains: true });
// {
//   domain: 'subdomain.blogspot.com',
//   domainWithoutSuffix: 'subdomain',
//   hostname: 'subdomain.blogspot.com',
//   isIcann: false,
//   isIp: false,
//   isPrivate: true,
//   publicSuffix: 'blogspot.com',
//   subdomain: ''
// }

There are a few other options that one can use to customize the behavior of tldts which you can find documented here: https://github.com/remusao/tldts/tree/master/packages/tldts#api
I hope this helps,
Best,

Very helpful, thanks for clarifying!


feature: Integrate FHIRPath evaluation for Feature set

Feature Request
Describe the Feature Request
FHIR Path evaluation should also work for variables that are created in featureset.
Is your feature request related to a problem? Please describe
No.
Describe Preferred Solution
Integrate FHIRPath evaluation for Feature set

Closed by merge request #18


remove server support for SSLv2Hello within jetty Edit

With the OpenSSL changes in CentOS 6.5, the client often fails when contacting the server. To avoid this problem, the server configuration should be changed so that support for SSLv2Hello is dropped.

This is not relevant anymore.
SSLv2 is deprecated and SlipStream now uses nginx to handle the SSL part.


Slide event cancellation doesn't keep previous slider values

I don't know if this project is still alive, but anyway according to the JQuery UI Slider docs you can cancel an slider event in the slider callback and keep the values that were just before the user started sliding.
(This is the documentation http://api.jqueryui.com/slider/#event-slide)
However if you use angular-ui-slider and try to cancel the slider event in the callback in the slider options, the values will change as if the event was never cancelled. After hours of debugging I found that the culprit is this listener which updates the model value from slider:
                // Update model value from slider
                elm.bind(updateOn, function(event, ui) {
                    var valuesChanged;

                    if (ui.values) {
                        var boundedValues = ui.values.slice();

                        if (options.lowerBound && boundedValues[0] < options.lowerBound) {
                            boundedValues[0] = Math.max(boundedValues[0], options.lowerBound);
                        }
                        if (options.upperBound && boundedValues[1] > options.upperBound) {
                            boundedValues[1] = Math.min(boundedValues[1], options.upperBound);
                        }

                        if (boundedValues[0] !== ui.values[0] || boundedValues[1] !== ui.values[1]) {
                            valuesChanged = true;
                            ui.values = boundedValues;
                        }
                    } else {
                        var boundedValue = ui.value;

                        if (options.lowerBound && boundedValue < options.lowerBound) {
                            boundedValue = Math.max(boundedValue, options.lowerBound);
                        }
                        if (options.upperBound && boundedValue > options.upperBound) {
                            boundedValue = Math.min(boundedValue, options.upperBound);
                        }

                        if (boundedValue !== ui.value) {
                            valuesChanged = true;
                            ui.value = boundedValue;
                        }
                    }

                    ngModel.$setViewValue(ui.values || ui.value);
                    $(ui.handle).find('.ui-slider-tip').text(ui.value);
                    scope.$apply();

                    if (valuesChanged) {
                        setTimeout(function() {
                            elm.slider('value', ui.values || ui.value);
                        }, 0);

                        return false;
                    }
                });

which is called regardless the slide event was cancelled or not; and, in fact, it's called before the callback supplied in the options.
I think the proper fix for this is to move that listener to a callback that is always supplied in the underlying JQuery UI widget that in turn calls the actual callback specified in the angular directive and only updates the view and triggers a digest if the event was not cancelled.
What do you think?
Cheers.

A workaround for this issue is to pass a custom event in the updateOn option and trigger it in the slider callback only when you don't cancel the event. For instance:
HTML:

Javascript (angular controller):
...
$scope.onSlide = function(event, ui){
  if(shouldCancelSlide(ui)){
    return false;
  }
  var slider = angular.element(event.target);
  slider.trigger('mySliderUpdate', ui);
};
...

I've tested this in the app I'm working on and it works :D

if this project is still alive

This project is kept alive only via contributions like yours. I merge PRs, but I can't do much more than that. If you think your fix should be implemented, please submit a PR, and I'll merge it.


Add an option to download releases.

[feature request]
I see I can list releases now, but I cannot download them from private repositories with just the URLs.
~/bin/hub release show 0.0.0-gpratt4 --show-downloads

## Downloads

https://github.com/topsecretorg/foobar/releases/download/0.0.0-gpratt4/foobar-linux-3.19.0-30-generic-x86_64-0.0.0-gpratt4.tar.gz
https://api.github.com/repos/topsecretorg/foobar/zipball/0.0.0-gpratt4
https://api.github.com/repos/topsecretorg/foobar/tarball/0.0.0-gpratt4

See #1103

Merged in e046b8d

@glennpratt Thanks for following up by closing this. In the future, you can use the phrase Fixes/closes #123 in your pull request body. That will automatically close the referenced issue when the PR is merged! :wink:


Is it possible to load .npy spectrograms directly?

Hi, on the notebook it isn't clear whether you can load your own spectrogram directly (for TTS inference) or not. Is this possible and if so, have you tried it?

Do you mean by "your own spectrogram" a mel-spectrogram from another speaker? Trained WaveGrad can take any mel-spectrogram as input: it should be of type 'torch.Tensor' and it should match STFT parameters, which were used to train WaveGrad. Generally, the output quality can depend on a speaker you are trying to feed the model with. I haven't test WaveGrad on unseen speakers, but I believe it should perform well.

Thanks! No, I meant feeding WaveGrad a .npy of a TTS trained on the same speaker, instead of running inference on the test set, because I didn't see it anywhere on the notebook (or I just missed it)

Yeah, if you have your mel of type np.array saved on a disk, than just load it and convert it into torch.Tensor. For conversion you can use classical torch.from_numpy(your_numpy_mel) or just torch.FloatTensor(your_numpy_mel). Finally, make sure that your mel has batch dimension, e.g. 1 x n_mels x n_frames and is on the same device as WaveGrad (CPU or GPU). Then, you can feed it as input to the forward method.


Don't fail if dataset is already destroyed

It is possible that a snapshot is detroyed by outside processes in the period where the filesystem and snapshot cache gets updated and currently zfssnap errs out, both by external destroy commands or deferred destroys. This is not necessary as we are happy either way if the snapshot is gone.
Also: should snapshots which are already marked for deferred destroy be counted when handling retention. I think not as the decision are already made for that snapshot?

Fixed by 37bd10d


Async validation on mount

Hello! Thanks for the awesome library.
I want to run the async validation on mount when the initial values are provided. But in this case the first argument of the async validation function is undefined. Please help me, this is the bug or maybe I am doing it wrong.
P.S. I use the latest version of redux-form (4.1.2).
JS Bin: https://jsbin.com/bijepudoxu/edit?js,console,output you will see undefined in Console tab.

componentWillMount() is too soon. I'm not 100% sure which part of the set up process hasn't happened yet, but if you just wait until the next process tick, it's fine.
  componentWillMount() {
    setTimeout(()=> {
      this.props.asyncValidate();
    });
  }

In practice you won't ever want to call asyncValidate() that early.
Thank you for the brilliantly explained issue, by the way. It was easy to click into JSBin, verify and rectify.


Hide the project tab in profiles and orgs

Fixes https://github.com/sindresorhus/refined-github/issues/984
Fixes #1757
Test
The tab is hidden if:

it's 0, and
it's not the current page, and
you can't disable projects

The "New project" is added if:

the project tab normally appears, and:

repos: all
orgs: only if you belong to the organization and thus can create projects

Try

Repo with 0 Projects tab: https://github.com/bfred-it-obsolete/test
Repo with projects: https://github.com/babel/babel
Organization: https://github.com/bfred-it-obsolete
Organization with projects: https://github.com/rust-lang
Organization you belong to ("New project" will appear)
Organization you own (the tab will not be hidden)
Profile: https://github.com/bfred-it
Your own profile (the tab will not be hidden)

Your own profile (the tab will not be hidden)

It is being hidden.

Ah yes, then it should be 🤣 We can't disable that tab, but also we don't want to see it there either.

The project tab is hidden on my own profile. Was that intended? If yes, I want to add my vote to show it again on the users own profile.


[test-framework] Add health check to local envoy agent at startup

Also fixing a bug in the envoy yaml template that was preventing the
configuration of multiple ports to a single instance.  Expanded the
test to cover this.

[APPROVALNOTIFIER] This PR is NOT APPROVED
This pull-request has been approved by:
To fully approve this pull request, please assign additional approvers.
We suggest the following additional approver: geeknoid
Assign the PR to them by writing /assign @geeknoid in a comment when ready.
The full list of commands accepted by this bot can be found here.
The pull request process is described here

Needs approval from an approver in each of these files:

pkg/OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@nmittler: The following test failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

prow/istio-unit-tests.sh
55fabc2ddb9f6d4d81111612e77e35f8e649fc2e
link
/test istio-unit-tests

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@ozevren PTAL

@ozevren also note that in my last commit, I moved the proxy code to the "local" package ... I think that makes more sense.

/lgtm
/approval

[APPROVALNOTIFIER] This PR is NOT APPROVED
This pull-request has been approved by: ozevren
To fully approve this pull request, please assign additional approvers.
We suggest the following additional approver: geeknoid
Assign the PR to them by writing /assign @geeknoid in a comment when ready.
The full list of commands accepted by this bot can be found here.
The pull request process is described here

Needs approval from an approver in each of these files:

pkg/OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@nmittler: The following test failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

prow/istio-unit-tests.sh
09396135999eb51d52323047ac6c3ea9c78b0f84
link
/test istio-unit-tests

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

New changes are detected. LGTM label has been removed.

@nmittler: The following test failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

prow/istio-unit-tests.sh
7406d712f0571fc79083515c1c2a7833cf19cfaa
link
/test istio-unit-tests

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.


branch name of the commits where the secrets detected

Is there a way to get the branch name of the commits where the secrets detected? Also Is there a way to have one more field that tells, if the secret is currently in github or its past commits?(means now its removed)
cc @zricethezav

@arunanandhan no, gitleaks does not expose branch names since gitleaks operates on commits and commits can be associated with many branches.

Also Is there a way to have one more field that tells, if the secret is currently in github or its past commits?(means now its removed)

If commit is set in the report or output that means the secret is in the git history and will be forever, or until you rewrite your git history (note that rewriting git history isn't a good solution to prevent secret leaks as copies containing secrets might still be out in the wild)


Add Array.contains, List.contains, and Seq.contains to TOC

/cc @liboz

Is that the file behind the left hand tree on the docs? If so, I will make sure any future commits I do change that too.

Yep, this is what lists them in the file tree.  No worries!  It's not obvious at all.  We should probably update the guidance to also add new files created to the TOC.


Add key to

React recycles the image tags, but when switching the src Chrome (and possibly other browsers?) will keep the old pixels visible until the new resource is loaded and rendered. This means you get a weird inconsistency as the images load in.

Hmm this didn't seem to totally solve the issue, but the code cleanup is awesome.
Thanks for this, ill merge it, push it live and we can see what else we need to do fix this.

This is live now :)


[CI] All OCL backend tests are disabled - do we need OCL backend?

#1693 disables all OCL tests, including smoke tests (I've overlooked that when reviewing). Is it worth to keep a bunch of untested code in the library? Let's be consistent and:

either remove the OCL backend totally
or enable smoke tests for it (FP32 at least).

I think we need some decision ASAP.

[Attribution] @junliume @johnny-keker https://github.com/ROCmSoftwarePlatform/MIOpen/labels/urgency_high https://github.com/ROCmSoftwarePlatform/MIOpen/labels/value_unknown, Proposed assignee: @JehandadKhan

I suggest we remove it, it is an unnecessary complication of our library as well as our CI. I am not aware of anyone using it at the moment

@DrizztDoUrden What do you think?

TBH removing it would ease a lot of headaches for me with serializing binaries, so I am kind of biased. (OCL can't get or load a binary without a stream handle afaik)

@junliume The ticket is closed, but I am not aware of the final decision. Can you please remind me which one of these two alternatives was implemented and in which PR?

Let's be consistent and:

either remove the OCL backend totally
or enable smoke tests for it (FP32 at least).


Temp

Affects: <jenkins-collector-version-number>.

Individual Agreement
Corporate Agreement

[ ] I Have signed the CLA

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.Bhagyashree S seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.You have signed the CLA already but the status is still pending? Let us recheck it.

Also I feel TeamCity is a different tool so that should be a seperate collector all together?

Sorry I raised that PR by mistake
On Thu, Apr 9, 2020, 8:46 PM Ayush<EMAIL_ADDRESS>wrote:

Also I feel TeamCity is a different tool so that should be a seperate
collector all together?
—
You are receiving this because you modified the open/close state.
Reply to this email directly, view it on GitHub
https://github.com/Hygieia/hygieia-build-jenkins-collector/pull/26#issuecomment-611584402,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ACMC2A5XDFFVC24V2YNICKLRLXRDTANCNFSM4MEYI6RQ
.


Redirects: Only relative paths are supported

The code at https://github.com/networkteam/Networkteam.Neos.ContentApi/blob/main/Classes/Controller/DocumentsController.php#L325 always prepends redirect targets with a slash, therefore absolute paths (e.g. different domains) are not supported.

Fix in https://github.com/networkteam/Networkteam.Neos.ContentApi/pull/10


Add livenessprobe to OneAgent from image

Description
Similar to the readinessProbe now the livenessProbe is set by checking depending if the Healthcheck is set in the dynakube status. Here, the livenesProbe will be set only if the readinessProbe is not nil.
How can this be tested?
Deploy the operator, apply a dynakube where oneagents spawn and check their status, if the livenessProbe is set or not.
Checklist

[x] Unit tests have been updated/added
[x] PR is labeled accordingly with a single label
[x] I have read and understood the contribution guidelines

Codecov Report

Merging #2115 (6c9b58d) into main (94aa287) will increase coverage by 0.01%.
The diff coverage is 94.44%.

:exclamation: Your organization needs to install the Codecov GitHub app to enable full functionality.
@@            Coverage Diff             @@
##             main    #2115      +/-   ##
==========================================
+ Coverage   63.24%   63.25%   +0.01%
==========================================
  Files         242      242
  Lines       15373    15378       +5
==========================================
+ Hits         9722     9727       +5
  Misses       4909     4909
  Partials      742      742

Files Changed
Coverage Δ

src/api/v1beta1/dynakube/properties.go
39.59% <0.00%> (ø)

...ntrollers/dynakube/oneagent/daemonset/daemonset.go
96.32% <100.00%> (+0.07%)
:arrow_up:


Change Transformers export default sequence length to max_position_embeddings

Usage
sparseml.transformers.export_onnx --model_path ./TinyLlama-1.1B-Chat-v0.3 --task text-generation --trust_remote_code
2023-11-13 18:56:23 sparseml.transformers.export INFO     Attempting onnx export for model at /home/mgoin/models/TinyLlama-1.1B-Chat-v0.3 for task text-generation
2023-11-13 18:56:23 sparseml.transformers.export INFO     Using default sequence length of 2048
Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.

Reason
Unfortunately I think it is very important to export LLMs with the largest sequence length possible.
For instance I was exporting llama models with a sequence length of 512 for convenience
sparseml.transformers.export_onnx --model_path ./llama2.c-stories15M --task text-generation --sequence_length 512
However due to the PyTorch export and constant folding it seems to produce a cached, fixed rotary embedding of the sequence length used. See it as 512 in this screenshot. The ONNX can still run and compile with a sequence length larger than 512, but the output is unstable and quickly starts to repeat

max_position_embeddings

The config class itself has a default value for this, so I'm inclined to leave it be. We could raise ValueError but that will be reflected downstream


ci/install.sh: Fix typos

Self-explanatory.

Can one of the admins verify this patch?

Codecov Report

Merging #63 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff            @@
##             master     #63   +/-   ##
========================================
  Coverage      80.5%   80.5%
  Complexity      625     625
========================================
  Files           114     114
  Lines          3308    3308
  Branches        182     182
========================================
  Hits           2663    2663
  Misses          543     543
  Partials        102     102

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 7d7236e...8028297. Read the comment docs.


access-om2: Added link to git repo, removed hardcoded version string

In this PR, we remove the hardcoded version("2023.0.001-1") and add in the git repo, so we can associate a particular build of access-om2 with a git commit (aka, so we can have a spack install<EMAIL_ADDRESS>for example).

This change makes it simpler/easier to update to a new version of the package because it doesn't require a change in this SPD?
I suppose that has significant advantages for our CI/CD testing pipeline as it doesn't require rebuilding and deploying the dependency containers as frequently?

Yes. Especially for CD as right now every version of ACCESS-OM2 will be 2023.01.001-1 to spack. The other alternative was to not use the ACCESS-OM2 SBD entirely, instead having the spack.yaml being a set of the dependencies already noted in the SBD.

Hi @harshula Alternatively, we always give an explicit version on installation I.e<EMAIL_ADDRESS>I'm against the idea of defaults personally. Either the default changes or it remains static.
If the default is dynamic, users may install the default version every time not knowing it might possibly change, or what the default version is.
If the default is static...is there such thing as a sensible default that isn't just the latest released version? In that case, every deployment would require an update to the SBD. I don't really know what any other sensible defaults would exist that wouldn't require updates each time there was a release.

https://github.com/ACCESS-NRI/spack_packages/issues/32#issuecomment-1794014603

How about using a string like latest or newest. We could then use the same string in the SPDs.


make stream state transitions observable on SendStream and ReceiveStream

Fixes #4428.

We'll go with #4445 and #4460 instead.


🐛[Bug Report]: Welcome screen Locale/Language and audio are not activated properly

Device model
All
OS version
Android 14 | Level 34 (Default)
Which version of Litewallet are you running?
Other
Describe what happened.

Go to ...
Tap on ...
Error, error! 🦑🔥

Add any notes you would like to add.
Problem
When the first time user sees the welcome screen the locale selection is not persisting when they open the app.  So, they are neveer able to use their locale language. This only happens for first time users
The key is that sometimes the feature works, but the picker sometimes is stuck in a state where the language is not set and defaults to English....this is bad
Regression
This worked previously and is was highly effective in getting users to onboard and use Litewallet.
The good news is this a relatively new feature so it shouldnt take long to understand the business logic
Screenshot Clip

Clip

Definition of Done

[ ] Ensure the welcome screen plays the local language and when the user selects it, the app shows that locale
[ ] Ensure that when the existing users are changing the language, it effectively switches to that language
[ ] PR should explain a description of where the bug caused the issue and a how it was fixed
[ ] Add 2 Tests to further protect the work and business logic

about audio not playing, maybe the audio not downloaded yet since using play asset delivery

Closing...this was resolved in v2.12.4 20241130 . Thanks @andhikayuana


Having array in QueryStringParameters

Hi Tim,
This is more a question than an issue with your code.
I successfully used your app with Client.GetJson((Resource)) (while Resource is the string appended to the API URL) to get information from TODOist API. This is the posted URL:
GET https://todoist.com/API/v6/sync?token=MY_TOKEN_HERE&seq_no=4595849276&seq_no_global=4595849276&resource_types=["projects"]

Now I'm trying to do the same, while making use of the WebRequest functions, but I'm stuck in the resource_types= parameter, as it asks for an array.
I tried adding this to the body, instead of the querystring, but the API won't accept.
Bottom line, is there a way to have arrays appended to the querystring, rather than the body?
Below is a copy of the code that is NOT working:
Dim Request As New WebRequest
    Request.Resource = "sync"
    Request.Method = WebMethod.Httpget
    Request.Format = WebFormat.Json

 Dim Body As New Dictionary
    Request.AddQuerystringParam "seq_no", 0
    Request.AddQuerystringParam "seq_no_global", 0
    Request.AddQuerystringParam "token", cfg.Project.Token

    Body.Add "resource_types", Array("projects")
    Set Request.Body = Body

    Set Response = cfg.Project.Client.Execute(Request)

Thanks for any help with that!
Mauricio

I intentionally left out array support in querystrings and url-encoded as there is no accepted standard (and the generally used form doesn't match TODOist, so it probably wouldn't work anyways).
For you case, I would try the following:
Dim ResourceTypes As String
ResourceTypes = "[""" & Join(Array("projects"), """, """) & """]"
' or directly with "[""projects""]"
Request.AddQuerystringParam "resource_types", ResourceTypes

http://api.jquery.com/jquery.param/ (Includes notes and traditional setting)
http://stackoverflow.com/questions/6243051/how-to-pass-an-array-within-a-query-string

In both the links you mention, there are references to arrays in the string. Is there an equivalent of the "serialize" in your library? Besides this [projects] call there are a number of other commands I should pass in the same format.


New English curriculum titles are not displayed appropriately in timeline

Describe the Issue
In the timeline table on the profile page, the title of each task of the New English curriculum is now displayed as Task 3, but each of the three courses contains a task with the same name, so campers can not recognize at a glance which course's task have been completed.
Affected Page
/username
Steps to Reproduce

Go to profile page after solving a task of new English curriculum
Scroll down to the table under the Timeline heading
See the curriculum title

Expected behavior
Like the Steps in new RWD or new JS Algorithms and Data Structures Certification, the title will be displayed with block name title and a dash.
Screenshots

The title of each task of New English Curriculum on the timeline table

System

Device: Laptop
OS: Windows 11 Home
Browser: Chrome
Version: 121.0.6167.161 (Official Build) (64-bit)

Additional context
No response

Thanks for pointing this out @mmatsumoto1026. I think that solution works. We want to make it so the English challenges display the block and the challenge title in the same way the RWD and new JS do. One issue with that is that we would get a pretty long title on those dialogue challenges. e.g. In the image above "Dialogue 1: Maria Introduces Herself to Tom" would probably become "Learn Greetings in your First Day at the Office - Dialogue 1: Maria Introduces Herself to Tom". I think that's fine though. I'm going to add the help wanted label.

Im going to take a look at this one 👀

Is there still anyone working on this?

Thanks for pointing this out @mmatsumoto1026. I think that solution works. We want to make it so the English challenges display the block and the challenge title in the same way the RWD and new JS do. One issue with that is that we would get a pretty long title on those dialogue challenges. e.g. In the image above "Dialogue 1: Maria Introduces Herself to Tom" would probably become "Learn Greetings in your First Day at the Office - Dialogue 1: Maria Introduces Herself to Tom". I think that's fine though. I'm going to add the help wanted label.

So, just to clarify: Task 1 in the block 'Learn Greetings in your First Day at the Office' then would be 'Learn Greetings in your First Day at the Office - Task 1', is that correct interpretation ?

Heyy @moT01 , I would like to work on this. I have setup the gitpod environment using the contribution guide.
But, I am facing a issue. when I try to open the profile page for the default logged in test account. It shows the Gatsby.js development 404 page. Can you please help me with this. Is there some env config i need to change to fix this ?

Thanks for your interest in this issue.

But, I am facing a issue. when I try to open the profile page for the default logged in test account. It shows the Gatsby.js development 404 page. Can you please help me with this. Is there some env config i need to change to fix this ?

When I faced a similar issue before, I was able to resolve it by referring to this comment.


2.x Fix Menu theme locations array to enforce string or integers

Issue
With Timber v2 , WPML, and PHP 8, if any nav menu location happens to be null, this warning occurs:
Warning: array_flip(): Can only flip string and integer values, entry skipped in vendor/timber/timber/src/Menu.php on line 255

This is similar to issue #2576 where the warning was with MenuFactory::get_menu_location().
Solution
The solution is to apply the same fix made to MenuFactory; filter the nav menu locations array to enforce strings or integers.
Testing
The TestTimberMenu::testThemeLocationProperty() method was updated to match TestMenuFactory::testGetMenuLocation().

Coverage: 88.685% (+0.003%) from 88.683% when pulling 781b8b8a30d52fa82d4bb0a44caaebc7f3ef5edf on mcaskill:2.x-fix-menu-theme-locations into c96b9f8f73e5fba3c6123f491a73734983b6b200 on timber:2.x.


HID support on macOS

#2632 added support for HID keyboards, using libusb. It is currently only supported on Linux (because it just works).
If someone knows how to make it work on macOS, your help is welcome.
You must enable the feature for all platforms:
diff --git a/app/meson.build b/app/meson.build
index 7f675035..ab035bf8 100644
--- a/app/meson.build
+++ b/app/meson.build
@@ -72,7 +72,7 @@ if v4l2_support
     src += [ 'src/v4l2_sink.c' ]
 endif

-usb_support = host_machine.system() == 'linux'
+usb_support = true
 if usb_support
     src += [
         'src/usb/aoa_hid.c',

Then, if you manage to make it wok, please submit a PR.

Please test #3031.


Added .VSCode settings for applying DSC PSSA rules

Pull Request (PR) description
Added .VSCode PSSA Settings to apply DSC Specific rules.
This Pull Request (PR) fixes the following issues

Fixes #357

Task list

[x] Added an entry under the Unreleased section of the change log in the CHANGELOG.md.
Entry should say what was changed, and how that affects users (if applicable).
[ ] Resource documentation added/updated in README.md in resource folder.
[ ] Resource parameter descriptions added/updated in schema.mof
and comment-based help.
[ ] Comment-based help added/updated.
[ ] Localization strings added/updated in all localization files as appropriate.
[ ] Examples appropriately added/updated.
[ ] Unit tests added/updated. See DSC Resource Testing Guidelines.
[ ] Integration tests added/updated (where possible). See DSC Resource Testing Guidelines.
[ ] New/changed code adheres to DSC Resource Style Guidelines and Best Practices.

This change is

Codecov Report

Merging #358 into dev will not change coverage.
The diff coverage is n/a.

@@         Coverage Diff         @@
##            dev   #358   +/-   ##
===================================
  Coverage    98%    98%
===================================
  Files        27     27
  Lines      1953   1953
===================================
  Hits       1920   1920
  Misses       33     33


Introduce a Never type

Rust has a Never type which helps with expressions like exit(0), break and continue. Sway could also benefit from that type for the following:

break
continue
ASM blocks that revert. (e.g. fn revert()).

This seems to be somewhat required to solve https://github.com/FuelLabs/sway/issues/2876 and all the problems that showed up when I tried fixing the IR verifier here: https://github.com/FuelLabs/sway/pull/2880. We've had issues hiding behind a bug in the verifier.

Right, this was discussed at some point here: https://github.com/FuelLabs/sway/issues/1697 but we never did anything about it.

I guess the key point with this type is that we want it to coerce to any other type and we can do this in the type engine by allowing enum {} to cast to any other type.

This could just be a standard library type in the prelude defined as enum Never {} perhaps.

I'm looking at this again now due to #3362 and I think if we're going to solve the diverging problem (again) properly then this won't be enough -- it needs to be a proper builtin type and special-cased in the type checker unification.  But I'm not a type system specialist.  @emilyaherbert ?


porting_lib.swim.speed

Hi, i can't get the attribut for the swim boost and the height foot step.
I have the last update zenith-0.1.11-1.20.1.jar for Fabric
Thank you

Fixed in 0.2.0, make sure to update Zenith Attributes as well

Hi i just launch the game today and made an update, thank you ^^ it works fine now😘


feat: Add test, gitignore, and convert project to Maven

Added a new test to ensure a good base for future changes.
Created a .gitignore file to exclude unnecessary files from version control.
Converted the project to Maven for improved dependency management and build processes.

Thank you very much for your contribution!! That's my first pull request into one of my repos ever 🤩


added alphabet pattern 32

added pattern no 32 and 33

@copyNinja1219 , I hope that SWOC has allowed for the contributors to start working on their chosen projects and can start creating PRs. I am merging this PR, if it does not get counted for SWOC leaderboard, I will count it for the PatternHouse SWAG Pack. Do have a look at the discord channel of patternhouse under swoc to be eligible for the swag pack


Swagger URL broke

Link to https://mobilitydata.github.io/mobility-feed-api/SwaggerUI/index.html no longer works.

Fixed now, the setting was missing from the repository.


Update work_issue.md

Change "id" to "ID" except for those in code.

Proposed changes

Unreleased project version (optional)

Related issues (optional)

Thanks, @scjane :)


Consider including provenance with results in development mode

The documentation outlines that multiple sources are used, but it's unclear what the provenance might be for any particular result, or whether a particular operation will generate network traffic or reference the local pickle files. Suggest a dev mode with more provenance.

I don't under this issue.  Can you provide more details on what you mean?

As a consumer when a list of results come back, and one looks spurious, it's not clear where it came from. Dictionary.com? Who knows. Including some attributes or columns with the results that point users to the provenance of the result would let them decide what to do next. Probably never necessary for runtime workloads.

I understand your point now.  I could return a dictionary that has the sources as the keys. I will explore on how to do this and provide a list if requested.

I understand your point now.  I could return a dictionary that has the sources as the keys. I will explore on how to do this and provide a list if requested.


Error: Not a valid base64 encoded string length - even though Base64 string seems to be valid

Hi!
I have a very strange problem - after upgrading my ap to Expo@51 my Base64 functionality got broken for some reason.
I use Base64 for easier exchange of objects containing encrypted data. All of my entries are stored on server and got encrypted using<EMAIL_ADDRESS>in front-end (the same on which now there is an error).
Despite that, when trying to decode following Base64 string:
eyJ2IjoiaHlicmlkLWNyeXB0by1qc18wLjIuNCIsIml2IjoiOEUwQmRvcHkzWWVwQ1UvS3lwa0tySFNxd2hyYmtadDJVTU5qZXlBRDFNWT0iLCJrZXlzIjp7IjY5OmViOmVkOjkwOjZkOmRjOmUxOmQ0OmE0OjhiOjVlOjc4OmIzOjFlOjdmOmUyOjkzOjNmOmE1OjMwIjoiYUlUZUFpOTJZSnZSN3g1bXNaTGVldzVuN1h2RVA1Mjh4aE9RTitybXFpUXl6aURaMWx3OUdrUmZtSDRCMlVQcDFoYVU0ejJsZ2tZclZ0aWEvUk04MnNhNlVRYTV5RzN6N05jbU0vTDVmaW1KNDNFTUhrZGVmRzhGRTdpZXAzWjA1eFIxNUdFMkhXMDFNZ0FhUUtpbUhuc291ZEZ6N1l3UG8vN2srNm1pVHBVPSJ9LCJjaXBoZXIiOiJCaldxTi9xcnVGemFvTEpSVGNKWG9TVGx2elRIUGJvNGluMURYa1R5V09OSUhkRWhnaHJJd082YS9YT2xuaHZZIn0=

I get an exception which I have never seen in my life:
Error: Not a valid base64 encoded string length

What is weirder, when I use https://www.base64decode.org/ or browser's atob there is no problem with decrypting this string.
Can anyone point me to a solution to this problem?

I could not reproduce your claim.  Here is what I got on node 20.
% node
Welcome to Node.js v20.12.2.
Type ".help" for more information.
> var { Base64 } = await import("js-base64");
undefined
> var s = 'eyJ2IjoiaHlicmlkLWNyeXB0by1qc18wLjIuNCIsIml2IjoiOEUwQmRvcHkzWWVwQ1UvS3lwa0tySFNxd2hyYmt0adDJVTU5qZXlBRDFNWT0iLCJrZXlzIjp7IjY5OmViOmVkOjkwOjZkOmRjOmUxOmQ0OmE0OjhiOjVlOjc4OmIzOjFlOjdmOmUyOjkzOjNmOmE1OjMwIjoiYUlUZUFpOTJZSnZSN3g1bXNaTGVldzVuN1h2RVA1Mjh4aE9RTitybXFpUXl6aURaMWx3OUdrUmZtSDRCMlVQcDFoYVU0ejJsZ2tZclZ0aWEvUk04MnNhNlVRYTV5RzN6N05jbU0vTDVmaW1KNDNFTUhrZGVmRzhGRTdpZXAzWjA1eFIxNUdFMkhXMDFNZ0FhUUtpbUhuc291ZEZ6N1l3UG8vN2srNm1pVHBVPSJ9LCJjaXBoZXIiOiJCaldxTi9xcnVGemFvTEpSVGNKWG9TVGx2elRIUGJvNGluMURYa1R5V09OSUhkRWhnaHJJd082YS9YT2xuaHZZIn0='
undefined
> Base64.decode(s)
'{"v":"hybrid-crypto-js_0.2.4","iv":"8E0Bdopy3YepCU/KypkKrHSqwhrbkZt2UMNjeyAD1MY=","keys":{"69:eb:ed:90:6d:dc:e1:d4:a4:8b:5e:78:b3:1e:7f:e2:93:3f:a5:30":"aITeAi92YJvR7x5msZLeew5n7XvEP528xhOQN+rmqiQyziDZ1lw9GkRfmH4B2UPp1haU4z2lgkYrVtia/RM82sa6UQa5yG3z7NcmM/L5fimJ43EMHkdefG8FE7iep3Z05xR15GE2HW01MgAaQKimHnsoudFz7YwPo/7k+6miTpU="},"cipher":"BjWqN/qruFzaoLJRTcJXoSTlvzTHPbo4in1DXkTyWONIHdEhghrIwO6a/XOlnhvY"}'
>  Base64.encode(Base64.decode(s)) == s
true

And I got the same result on browsers.

That's exactly why I created this issue. In my code when I do const a = Base64.encode(...) and then Base64.decode(a) it works perfectly, and as I mentioned using some online decoders or browser's atob it also works.
I just wanted to know if you can think of any reason why such issue might occur? Maybe I should somehow enforce given encoding of input string?
I don't really understand this error message - I've checked that this string is 540 characters long (taken from this example and inside my code) which seems correct

for one thing Base64.decode() does not thrown an exception even when the input Base64 is invalid.  It just returns invalid data since any binary data can be encoded to Base64.  There is indeed Base64.isValid() that checks if the string is in correct Base64 format but as I say Base64.decode() does accept invalid strings.  So I can conclude there is something wrong in YOUR environment, not this module.

That's all I needed to know, thank you!

Just to follow up for anyone having the same issue as me - I still didn't resolve the root cause of the problem, but I'm quite sure it's caused by react native Buffer implementation. The issue appeared after upgrading from<EMAIL_ADDRESS>to 0.74.1.
Base64.decode under the hood does Buffer.toString() (as far as I can tell by examinig the code), which is why I suspect Buffer to be an issue, and also I can't really change it in any way.
I ended up using decoding algorithm posted here: https://en.wikibooks.org/wiki/Algorithm_Implementation/Miscellaneous/Base64#Javascript_2 - although this might not be as fast as default implementation, it works and doesn't use problematic Buffer

Thank you for your report.  If so the quick and dirty patch below should solve the problem.  Would you try and see what happens?
% git diff
diff --git a/base64.js b/base64.js
index dc837a7..04ea64d 100644
--- a/base64.js
+++ b/base64.js
@@ -42,7 +42,7 @@
      * @deprecated use lowercase `version`.
      */
     var VERSION = version;
-    var _hasBuffer = typeof Buffer === 'function';
+    var _hasBuffer = false; // typeof Buffer === 'function';
     var _TD = typeof TextDecoder === 'function' ? new TextDecoder() : undefined;
     var _TE = typeof TextEncoder === 'function' ? new TextEncoder() : undefined;
     var b64ch = 'ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/=';
diff --git a/base64.mjs b/base64.mjs
index fe9cfa5..a4bf113 100644
--- a/base64.mjs
+++ b/base64.mjs
@@ -14,7 +14,7 @@ const version = '3.7.7';
  * @deprecated use lowercase `version`.
  */
 const VERSION = version;
-const _hasBuffer = typeof Buffer === 'function';
+const _hasBuffer = false; // typeof Buffer === 'function';
 const _TD = typeof TextDecoder === 'function' ? new TextDecoder() : undefined;
 const _TE = typeof TextEncoder === 'function' ? new TextEncoder() : undefined;
 const b64ch = 'ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/=';
diff --git a/base64.ts b/base64.ts
index 845d1ee..797927a 100644
--- a/base64.ts
+++ b/base64.ts
@@ -14,7 +14,7 @@ const version = '3.7.7';
  * @deprecated use lowercase `version`.
  */
 const VERSION = version;
-const _hasBuffer = typeof Buffer === 'function';
+const _hasBuffer = false; // typeof Buffer === 'function';
 const _TD = typeof TextDecoder === 'function' ? new TextDecoder() : undefined;
 const _TE = typeof TextEncoder === 'function' ? new TextEncoder() : undefined;
 const b64ch =

I will try it today around 6-7PM (CET) and let you know

@mlazari Thank you! I will check this out asap

@PiotrSzlagura It's strange though, because the string you posted has a length that is divisible by 4 (540). As I understand the error was thrown for strings that have a length not divisible by 4 and the workaround in that link was to pad it with "=" at the end until its length is divisible by 4. Anyways, 0.74.2 fixed this for me.


More info on git_management please

_Originally posted by @Alphamasterliu in
https://github.com/OpenBMB/ChatDev/issues/136#issuecomment-1751914383_
Sorry, I also would like more information about what git_management actually does, I don't think your readme or wiki explains it very well. Would you be willing to expand on what git_management offers?
September 25th, 2023: The Git feature is now available, enabling the programmer to utilize GitHub for version control. To enable this feature, simply set "git_management" to "True" in ChatChainConfig.json.

Sorry for the confusion. We will update the wiki.

We have updated the wiki, you can see introduction at https://github.com/OpenBMB/ChatDev/blob/main/wiki.md#git-mode

@thinkwee Just to confirm, this might also be good to add on the readme, but the github push is only per project right? You can't have chatdev reopen a previous github repo it previously uploaded code to, right?

ChatDev can not open another git repository for the moment.  It can only add several git commits to the current software to be generated to facilitate backtracking and modification.

@thinkwee I don't mean another git repo, I mean if I rerun chatdev, could it reopen the same git repo it just created in the previous run? So I could further modify the script it made?

@thinkwee I don't mean another git repo, I mean if I rerun chatdev, could it reopen the same git repo it just created in the previous run? So I could further modify the script it made?

No, ChatDev can not reopen the same git repo because it will create a new folder with a new timestamp. However, ChatDev can be slightly modified to open an existing project and then continue with the git operation. Many users hope to have such an incremental development function, but we are still discussing the specific implementation, because everyone's specific needs for incremental development are very different, and we hope to be able to provide a more general solution.


Fix config.server_bundle_js file in generator

Change react_on_rails.rb to a thor template
Added server_rendering conditional
Added comment
Added tests

Fixes #294

Outstanding! @aaronvb Thanks! We need a list of React on Rails Heroes! You're on the list!

Reviewed 4 of 4 files at r1.
Review status: all files reviewed at latest revision, all discussions resolved, some commit checks failed.

Comments from the review on Reviewable.io


Fix native image pipeline document-intelligence

API change check
API changes are not detected in this pull request.