Parsing Error occurred when using Yip-Matthews Bilingual Corpus

Describe the bug
When I try to use the Yip-Matthews Bilingual Corpus, the following error occurs:
---------------------------------------------------------------------------
_RemoteTraceback                          Traceback (most recent call last)
_RemoteTraceback:
"""
Traceback (most recent call last):
  File "/opt/tljh/user/lib/python3.9/concurrent/futures/process.py", line 243, in _process_worker
    r = call_item.fn(*call_item.args, **call_item.kwargs)
  File "/opt/tljh/user/lib/python3.9/concurrent/futures/process.py", line 202, in _process_chunk
    return [fn(*args) for args in chunk]
  File "/opt/tljh/user/lib/python3.9/concurrent/futures/process.py", line 202, in <listcomp>
    return [fn(*args) for args in chunk]
  File "/home/jupyter-raptor/.local/lib/python3.9/site-packages/pylangacq/chat.py", line 1430, in _parse_chat_str
    utterances = self._get_utterances(all_tiers)
  File "/home/jupyter-raptor/.local/lib/python3.9/site-packages/pylangacq/chat.py", line 1449, in _get_utterances
    utterance_line = _clean_utterance(tiermarker_to_line[participant_code])
  File "/home/jupyter-raptor/.local/lib/python3.9/site-packages/pylangacq/_clean_utterance.py", line 195, in _clean_utterance
    utterance = _drop(utterance, "> [/]", "<", ">", "left")
  File "/home/jupyter-raptor/.local/lib/python3.9/site-packages/pylangacq/_clean_utterance.py", line 118, in _drop
    paren_i = _find_paren(
  File "/home/jupyter-raptor/.local/lib/python3.9/site-packages/pylangacq/_clean_utterance.py", line 112, in _find_paren
    raise ValueError(f"no matching paren: {s}, {target}, {opposite}, {direction}")
ValueError: no matching paren: see my babe [/] babe , <, >, left
"""

The above exception was the direct cause of the following exception:

ValueError                                Traceback (most recent call last)
Cell In[3], line 7
      5 url = "https://childes.talkbank.org/data/Biling/CHCC.zip"
      6 url = "https://childes.talkbank.org/data/Biling/YipMatthews.zip"
----> 7 corpus = pycantonese.read_chat(url)

File ~/.local/lib/python3.9/site-packages/pylangacq/chat.py:187, in _params_in_docstring.<locals>.real_decorator.<locals>.wrapper(*args, **kwargs)
    185 @functools.wraps(func)
    186 def wrapper(*args, **kwargs):
--> 187     return func(*args, **kwargs)

File ~/.local/lib/python3.9/site-packages/pycantonese/corpus.py:423, in read_chat(path, match, exclude, encoding)
    402 @_params_in_docstring("match", "exclude", "encoding", class_method=False)
    403 def read_chat(
    404     path: str, match: str = None, exclude: str = None, encoding: str = _ENCODING
    405 ) -> CHATReader:
    406     """Read Cantonese CHAT data files.
    407
    408     Parameters
   (...)
    421     :class:`~pycantonese.CHATReader`
    422     """
--> 423     return pylangacq_read_chat(
    424         path, match=match, exclude=exclude, encoding=encoding, cls=CHATReader
    425     )

File ~/.local/lib/python3.9/site-packages/pylangacq/chat.py:187, in _params_in_docstring.<locals>.real_decorator.<locals>.wrapper(*args, **kwargs)
    185 @functools.wraps(func)
    186 def wrapper(*args, **kwargs):
--> 187     return func(*args, **kwargs)

File ~/.local/lib/python3.9/site-packages/pylangacq/chat.py:1846, in read_chat(path, match, exclude, encoding, cls)
   1844 path_lower = path.lower()
   1845 if path_lower.endswith(".zip"):
-> 1846     return cls.from_zip(path, match=match, exclude=exclude, encoding=encoding)
   1847 elif os.path.isdir(path):
   1848     return cls.from_dir(path, match=match, exclude=exclude, encoding=encoding)

File ~/.local/lib/python3.9/site-packages/pylangacq/chat.py:187, in _params_in_docstring.<locals>.real_decorator.<locals>.wrapper(*args, **kwargs)
    185 @functools.wraps(func)
    186 def wrapper(*args, **kwargs):
--> 187     return func(*args, **kwargs)

File ~/.local/lib/python3.9/site-packages/pylangacq/chat.py:1127, in Reader.from_zip(cls, path, match, exclude, extension, encoding, parallel, use_cached, session)
   1124         with zipfile.ZipFile(zip_path) as zfile:
   1125             zfile.extractall(unzip_dir)
-> 1127     reader = cls.from_dir(
   1128         unzip_dir,
   1129         match=match,
   1130         exclude=exclude,
   1131         extension=extension,
   1132         encoding=encoding,
   1133         parallel=parallel,
   1134     )
   1136 # Unzipped files from `.from_zip` have the unwieldy temp dir in the file path.
   1137 for f in reader._files:

File ~/.local/lib/python3.9/site-packages/pylangacq/chat.py:187, in _params_in_docstring.<locals>.real_decorator.<locals>.wrapper(*args, **kwargs)
    185 @functools.wraps(func)
    186 def wrapper(*args, **kwargs):
--> 187     return func(*args, **kwargs)

File ~/.local/lib/python3.9/site-packages/pylangacq/chat.py:1057, in Reader.from_dir(cls, path, match, exclude, extension, encoding, parallel)
   1055             continue
   1056         file_paths.append(os.path.join(dirpath, filename))
-> 1057 return cls.from_files(
   1058     sorted(file_paths),
   1059     match=match,
   1060     exclude=exclude,
   1061     encoding=encoding,
   1062     parallel=parallel,
   1063 )

File ~/.local/lib/python3.9/site-packages/pylangacq/chat.py:187, in _params_in_docstring.<locals>.real_decorator.<locals>.wrapper(*args, **kwargs)
    185 @functools.wraps(func)
    186 def wrapper(*args, **kwargs):
--> 187     return func(*args, **kwargs)

File ~/.local/lib/python3.9/site-packages/pylangacq/chat.py:1009, in Reader.from_files(cls, paths, match, exclude, encoding, parallel)
   1006 else:
   1007     strs = [_open_file(p) for p in paths]
-> 1009 return cls.from_strs(strs, paths, parallel=parallel)

File ~/.local/lib/python3.9/site-packages/pylangacq/chat.py:187, in _params_in_docstring.<locals>.real_decorator.<locals>.wrapper(*args, **kwargs)
    185 @functools.wraps(func)
    186 def wrapper(*args, **kwargs):
--> 187     return func(*args, **kwargs)

File ~/.local/lib/python3.9/site-packages/pylangacq/chat.py:970, in Reader.from_strs(cls, strs, ids, parallel)
    966     raise ValueError(
    967         f"strs and ids must have the same size: {len(strs)} and {len(ids)}"
    968     )
    969 reader = cls()
--> 970 reader._parse_chat_strs(strs, ids, parallel)
    971 return reader

File ~/.local/lib/python3.9/site-packages/pylangacq/chat.py:254, in Reader._parse_chat_strs(self, strs, file_paths, parallel)
    252 if parallel:
    253     with cf.ProcessPoolExecutor() as executor:
--> 254         self._files = collections.deque(
    255             executor.map(self._parse_chat_str, strs, file_paths)
    256         )
    257 else:
    258     self._files = collections.deque(
    259         self._parse_chat_str(s, f) for s, f in zip(strs, file_paths)
    260     )

File /opt/tljh/user/lib/python3.9/concurrent/futures/process.py:559, in _chain_from_iterable_of_lists(iterable)
    553 def _chain_from_iterable_of_lists(iterable):
    554     """
    555     Specialized implementation of itertools.chain.from_iterable.
    556     Each item in *iterable* should be a list.  This function is
    557     careful not to keep references to yielded objects.
    558     """
--> 559     for element in iterable:
    560         element.reverse()
    561         while element:

File /opt/tljh/user/lib/python3.9/concurrent/futures/_base.py:608, in Executor.map.<locals>.result_iterator()
    605 while fs:
    606     # Careful not to keep a reference to the popped future
    607     if timeout is None:
--> 608         yield fs.pop().result()
    609     else:
    610         yield fs.pop().result(end_time - time.monotonic())

File /opt/tljh/user/lib/python3.9/concurrent/futures/_base.py:438, in Future.result(self, timeout)
    436     raise CancelledError()
    437 elif self._state == FINISHED:
--> 438     return self.__get_result()
    440 self._condition.wait(timeout)
    442 if self._state in [CANCELLED, CANCELLED_AND_NOTIFIED]:

File /opt/tljh/user/lib/python3.9/concurrent/futures/_base.py:390, in Future.__get_result(self)
    388 if self._exception:
    389     try:
--> 390         raise self._exception
    391     finally:
    392         # Break a reference cycle with the exception in self._exception
    393         self = None

ValueError: no matching paren: see my babe [/] babe , <, >, left

To reproduce

Execute the following codes:

import pycantonese
url = "https://childes.talkbank.org/data/Biling/YipMatthews.zip"
corpus = pycantonese.read_chat(url)

The above error appears.

Expected behavior
Expected the corpus can be used without error, just like Child Heritage Chinese Corpus, Guthrie Bilingual Corpus, HKU-70 Corpus, Lee-Wong-Leung Corpus, Leo Corpus and Paidologos Corpus: Cantonese.
All links are checked, only the Yip-Matthews Bilingual Corpus shows an error.
System (please complete the following information):

Operating System: Ubuntu 18.04
PyCantonese version: 3.4.0

Additional context
Running in Jupyterhub

Hello! It looks like the upstream CHILDES and TalkBank data has been updated/fixed. I've just checked that except for "Paidologos Corpus: Cantonese" (as of this writing, accessing https://phonbank.talkbank.org returns an error), pycantonese can load and successfully parse the datasets listed in the pycantonese documentation without crashing.
Because by default downloaded data is cached on your local drive, if you still use the same machine/system/etc. when you first created this issue, you may still have the previously downloaded yet "faulty" Yip-Matthews corpus copy on disk. To force re-downloading, rather than the convenience function read_chat() which doesn't expose many arguments, use CHATReader.from_zip() that has the boolean use_cached argument (default is True, and you'd want to set it to False in this case):
import pycantonese
url = "https://childes.talkbank.org/data/Biling/YipMatthews.zip"
corpus = pycantonese.CHATReader.from_zip(url, use_cached=False)

After you've used CHATReader.from_zip() for a given URL once, you can switch back to read_chat() for the same URL to use the cached data and skip re-downloading if you so choose.
Hope this helps! Closing this issue as resolved.


Broker selection fields on install

Problem
0.14 will come with two brokers:

channel based
multitenant channel broker

There are now different artifacts, that we need to take into account.
We have two convenient release yamls:

eventing.yaml: this have the the channel version
eventing-mt.yaml: this will have the multitenant version

However,  that means you pick one or the other.
Generally the artifacts we ship are:

eventing-crds.yaml
eventing-core.yaml
channel-broker.yaml
mt-channel-broker.yaml

How about we have the operator to apply the first two yamls, and based on installation choice (or later updated CR), we pick which broker we apply?
The default would be channel,
apiVersion: operator.knative.dev/v1alpha1
kind: KnativeEventing
metadata:
  name: knative-eventing
  namespace: knative-eventing
spec:
  broker: channel

Persona:
Which persona is this feature for?

System integrator

Exit Criteria
installation of broker done, based on broker field
Time Estimate (optional):
How many developer-days do you think this may take to resolve?
Additional context (optional)
Add any other context about the feature request here.

@matzew So far, the manifest is loaded before creating the CR.
To make CR configuration able to determine how to load the manifest:
One way is to load different manifest in the controller, and use a map with key-value, maintaining in the reconciler. By different CR broker configurations, reconciler can select the correct manifest, with the key of the map.
The other way is to dynamically load the manifest in reconciler. Each time reconciling loop is kicked-off, load the manifest at this beginning, based on CR.
@jcrossley3 What do you think?

What about we add the validation webhook in eventing-operator?
The old value of the CR is saved in the context.
Then, implement the function Validate for the eventing CR, similar to https://github.com/knative/serving/blob/master/pkg/apis/serving/v1beta1/route_validation.go#L35 to retrieve the existing value, when it is an update.
If the new value for broker is the same as the existing one, we move on. If not, raise an error, stating that Users need to stay with the old broker. If u want to change, you need to remove and recreate the CR.

As I discussed with @n3wscott. All the channels can be installed and run at the same time without influencing each other, we do not necessarily need to delete other channels, before installing the channel we select.
@jcrossley3 @matzew @aliok

Not exactly sure what this broker field has to do with the channels.
But there is a some difference in the way the other channels come and bundle their yamls.:
None of the other channels, come w/ a ConfigMap (config-br-defaults) override to set themselves as default.
The brokers both, come with the same configmap, with different values:

https://storage.googleapis.com/knative-nightly/eventing/latest/mt-channel-broker.yaml
https://storage.googleapis.com/knative-nightly/eventing/latest/channel-broker.yaml

both have them...
so if you apply the first, than the second, they do have effect.
Channels do not do that....
No config maps in here, which does SET the channel as default

https://github.com/knative/eventing/tree/master/config/channels/in-memory-channel
https://github.com/knative/eventing-contrib/tree/master/natss/config
https://github.com/knative/eventing-contrib/tree/master/kafka/channel/config

What I understand is:

Having both brokers at the same time is possible
mt-channel-broker.yaml and channel-broker.yaml has some resources twice.
** Some of them are conflicting, like config-br-defaults configMap, which sets the mtbroker or the channelBroker as the default broker
** Some of them are simply identical

With @matzew 's PR, https://github.com/knative/eventing/pull/2936, non-conflicting stuff is going to be in the YAML and the conflicting stuff will be separate, so that they can be applied invididually.
e.g., apply the release yaml, then

apply this and you have the ChannelBasedBroker as the default
apply this and you have the MTChannelBasedBroker as the default broker.

I like the idea of having broker field in the CR. The value of that should the brokerClass names that are possible to pass into default-br-config. So, ChannelBasedBroker and MTChannelBasedBroker for now, and maybe others in the future.

/assign @aliok
I have the bandwidth to work on this one

Thanks @aliok

I suggest the field to be called

defaultBrokerClass:  consistent with the default-br-config (https://raw.githubusercontent.com/knative/eventing/2c1c5c2e2e7cbb36a716e6fca8802f71704775ff/config/400-config-br-defaults.yaml)
OR
defaultBroker

Ex:
apiVersion: operator.knative.dev/v1alpha1
kind: KnativeEventing
metadata:
  name: knative-eventing
  namespace: knative-eventing
spec:
  defaultBrokerClass: ChannelBasedBroker

I see brokerClass, not defaultBrokerClass.


Fix build script

I was able to use pump instead of pipes to get more details on the real error during all of the streaming. Uglify doesn't support minifying ES6. I am not familiar with Browserify, but I think it is pulling in node libraries that have const in them, and it isn't transforming those with babel, just the source code. Maybe an older version of browserify or maybe node(?) would allow this build to work without this patch.
Since we are building a chrome extension, const and other ES6 is fine to leave in the production code. uglify-es is now used for minification. It handles ES6.

If you don't like this, I think we'd be looking at switching off browserify. Let me know if you feel that's worth it.

@dylang bump


Package: add package url

I would like to introduce package_url to packages which will hold the url to the package details.
I tried to introduce it for every package manager, could not find a package registry for carthage and stb packages.
In a second step I would like to enable csv export of the package_url as a new column. An additional PR for this will follow when this one will be merged.

@forelabs Thanks again for your PR! I am curious if the homepage field satisfies this criteria. I know that that field comes from what is found in the package manager itself and I am just wondering if these two fields would be used differently but users or if they are just different ways to get the same information?

@xtreme-shane-lattanzio I think those are different. E.g. for jquery homepage is https://jquery.com/ but it is not that easy to get all relevant information from this site like verify license, see authors, ...
To provide an url where one could fetch easily this data from we need to provide the package url.
If this is special for us I am fine not getting this into the gem but I would guess that this information might be relevant for some people - at least a few of our customers would like to get this information to verify our fetched data.

@forelabs Is it intentional that the package_url field for Bower and yarn aren't version specific? We noticed that all the other package managers are but these 2 were not.

@forelabs Is it intentional that the package_url field for Bower and yarn aren't version specific? We noticed that all the other package managers are but these 2 were not.

I would like to have version specific urls for those too but actually the registries don't provide that possibility. The versions are visible as lists on those sites.

@forelabs I found this where they mention a curl command -
curl https://registry.bower.io/packages/<package_name>

This is providing a json response with a proper URL to the package info. Do you think this would be a better solution to your use-case and also a better alternative to the current search api call to Bower?

@pivotal-pmital Thank you for that, since bower relays on github packages only I think this is a good way. I have changed implementation to fetch url from the registry endpoint.


Coverity scan issue - Dereference nil return value

GORM Playground Link

https://github.com/go-gorm/playground/pull/1
Description
This has been caught as part of static analysis tool report, version mentioned below:
error - Dereference nil return value
function - DB.Association
details - If the function actually returns a nil value, a nil pointer dereference will occur. In github.com/jinzhu/gorm.DB.Association(string, *github.com/jinzhu/gorm.Association): Return value of function which returns nil is dereferenced without checking (CWE-476)

version - github.com/jinzhu/gorm v1.9.12

Hi @jinzhu , this is an issue which is an static analysis tool report, thought of sharing here to get insights for the issue and possible discussion to fix this.
Unfortunately, in this case i do not have go playground to simulate, as this scan report of the code base


Full regen of MGI; First load of wb_oa_3.gpad

Drop and reload of 13489 MGI models.
Overwrite of 155 WB models from wb_oa_3.gpad. Any pre-existing WB models for genes not annotated in wb_oa_3.gpad were untouched in this commit.
Fixes missing Assigned By issue geneontology/gocamgen#89.
Adds gene symbol to model title - geneontology/gocamgen#90
No errors during gocamgen translation for either MGI or WB.
go_cam_mgi.gpad header:
!gpa-version: 2.0
!generated-by: WormBase
!date-generated: 2021-05-13

wb_oa_3.gpad header:
!gpa-version: 2.0
!generated-by: MGI
!date-generated: 2021-05-14

Tagging @vanaukenk @ukemi

Taking on faith of @dustine32


CASE namespace concept

Define 3 namespace types, where 1 & 2 are maintained by CASE dev team:

CASE (case.example.org) - CASE-approved property bundles, promoted from Community namespace by voting body (Official Namespace)
CASE Community (cc.example.org) - Property bundles promoted by individuals that would benefit the community
Individual (my company.com) - unique property bundles that would not necessarily benefit others outside your use cases

^ To clarify, we think this could help avoid having to use a miscellaneous/wrapper propertyBundle for gaps in the ontology that adopters identify as something that should not be in their custom namespace. We can do some statistics or voting on the usage of Community namespace objects to promote up to official CASE standard as usage increases/the objects gain support.


Automate the release of docker containers.

Add github actions to build and release containers for the different versions.

Tags will now be automatically released.
Fixed by #36 and updating the repositories permissions.


fix: do not let Chromium manage webview's frame's lifetime

Description of Change
This is part of fixes for https://github.com/electron/electron/issues/14211.
As explained in https://github.com/electron/electron/issues/14211#issuecomment-418616406, Chromium has some race conditions when managing WebFrame of webview that, it will sometimes accidentally detach the WebFrame of webview, resulting in the WebContents of webview being wrongly destroyed.
This patch disables Chromium's ability to detach the WebFrame of webview. There will be another patch in Electron to manually manage the WebContents of webview on Electron's side.
Checklist

[x] PR description included and stakeholders cc'd
[x] Patch information is added to appropriate .patches.yaml
[x] script/update runs without error
[x] PR title follows semantic commit guidelines

@zcbenz this also needs to be backported into the electron-3-0-x branch, i can do that this morning


Incorrect definition of 'med2'

The description of 'med2' does not match its description in the Syriac script development page or its implementation in Uniscribe. It currently says:

Function: Replaces Alaph glyphs in the middle of Syriac words when the preceding base character cannot be joined to.
Example: When an Alaph is preceded by a Waw, the Alaph would be replaced by an appropriate form.

It should say:

Function: Replaces Alaph glyphs in the middle of Syriac words when the preceding base character can be joined to.
Example: When an Alaph is preceded by a Heth, the Alaph would be replaced by an appropriate form.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 61a9106c-ca98-1891-2c00-ace78615994f
Version Independent ID: 0feefbc8-23a7-5bc8-76dc-0a336a1c7968
Content: Registered features, k-o - Typography
Content Source: typographydocs/opentype/spec/features_ko.md
Product: windows
GitHub Login: @PeterCon
Microsoft Alias: PeterCon

Contradictory statements, so clearly a problem. I suspect you are correct, but this can't be deduced simply by looking at how text displays with a certain font since it's not clear what lookups within the font actually led to that display.

Fixed for the next version.


Find and replace with regex does not work for "replace all instances"

Issue Report Checklist

[x] Searched the issues page for similar reports
[x] Read the relevant sections of the Spyder Troubleshooting Guide and followed its advice
[x] Reproduced the issue after updating with conda update spyder (or pip, if not using Anaconda)

[x] Restarted Spyder
[ ] Reset preferences with spyder --reset
[ ] Reinstalled the latest version of Anaconda
[ ] Tried the other applicable steps from the Troubleshooting Guide

[x] Completed the Problem Description, Steps to Reproduce and Version sections below

Problem Description
Using find and replace, I normally can change from 4 spaces to tabs by using the regex expression '\t'. In the last versions of spyder, this works good for "replace next occurrence" and "replace occurrences in selection" but not for "replace all occurrences". With "replace all", the spaces are replaced by the characters \t and not by a tabulation.
Occurs at least with 5.5.1 and 5.5.4. It used to work good with version 5.4.3 (to be verified the exact version)
What steps reproduce the problem?

Find and replace, with regex expression.
Spaces in the "find", and '\t' in the "replace"
Click on replace all occurrences.

 We'll take a look at this one after we release Spyder 6 in a couple of months (sorry, we're busy trying to wrap up that release).

Thank you. For further information, I checked that it also worked properly with version 5.4.5. So the problem happened somewhere between this version and 5.5.1

Ok, if this was working before then it's a regression. So, I'll try to fix it in our next version (5.5.5), to be released shortly.

Sorry, I'm going to leave this for 6.0.1 because I don't have time to take a look at it right now and it's a minor regression.


US_5.6: Group accepted user by color

As
organizer
I want to
group the participants of one event
in order to
split the participants in little work groups.
Acceptance criteria:

As organizer I want to label with an color specific user
As organizer I want to the the column "group" and there I can see rectangle colors for each participant
As organizer I want to sort by groups to generate name badges for example. (#33)

How much colors?


Remove or update the goatcounter.com URL

This spec is copied from the Target one, but keeps the goatcounter URL in the dev.html to count visitors.
https://github.com/kg-construct/collection-containers-spec/blob/main/dev.html#L209
Goatcounter is a privacy safe alternative for Google Analytics.
Please update the URL otherwise the visitor counter will be wrong soon :)

done


Store SemVerLevel for newly pushed/uploaded packages to gallery

When new packages are pushed/uploaded to the gallery, we should store the SemVerLevel in the gallery database.
Add new column “SemVerLevelId” of type int, hard-code the value for now, and have the ability to expand with another table later on when needed.
This requires minimal storage, provides optimal speed, and still allows us to be flexible in the event we need to support additional semver-levels.
ALTER TABLE [dbo].[Packages] ADD [SemVerLevelKey] [int]

Verified in prod by reflowing existing SemVer 2.0.0 package (semVerA).


fix: bugs

Codecov Report

Merging #12 into master will increase coverage by 0.35%.
The diff coverage is 69.44%.

@@            Coverage Diff             @@
##           master      #12      +/-   ##
==========================================
+ Coverage   26.11%   26.46%   +0.35%
==========================================
  Files          30       30
  Lines         762      767       +5
  Branches      174      175       +1
==========================================
+ Hits          199      203       +4
  Misses        560      560
- Partials        3        4       +1

Impacted Files
Coverage Δ

src/cli/bin.ts
0.00% <ø> (ø)

src/program/intermediate-type/create-template.ts
0.00% <0.00%> (ø)

src/program/intermediate-type/create-type-text.ts
0.00% <0.00%> (ø)

src/program/type-definition.ts
0.00% <0.00%> (ø)

src/typeshot/printer.ts
0.00% <ø> (ø)

src/typeshot/register-type-definition.ts
0.00% <0.00%> (ø)

src/utils/stack-tracking.ts
100.00% <100.00%> (ø)

src/utils/tagged-template.ts
100.00% <100.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 3891a04...32bf844. Read the comment docs.


File not ready to read after extraction

Firstly thank you for the library you have developed.
I'm using a progress for fraction completed, when this is 1 I open the file but the file is corrupt. I've put a delay in the code to wait 3 seconds after it reports the fractionCompleted as 1 and it works as a temporary hack. I'm wondering if the fractionCompleted is reporting 1 to the progress before the file is 100% ready to read or is there a better way to trigger and action using the file when it has been unzipped?
self.zipProgress = Progress()
self.zipProgress?.addObserver(self, forKeyPath: "fractionCompleted", options: .new, context: nil)
try archive.extract(entry, to: destinationURL, progress: self.zipProgress)

The totalUnitCount for the progress is in bytes. Can you check if completion is reported correctly if you use the convenience method Archive.makeProgressForReading(_ entry: Entry). This is a shortcoming of the current progress API. It does not enforce correct unit counts.

I think that method must be internal so I can't call it from outside the class. So I tried
self.zipProgress = Progress(totalUnitCount: archive.totalUnitCountForReading(entry))

Same error of corrupt data when I try to read the extracted file.
Just in case my code has a mistake at the other end, I'll list it here.
override func observeValue(forKeyPath keyPath: String?, of object: Any?, change: [NSKeyValueChangeKey : Any]?, context: UnsafeMutableRawPointer?) {
        if keyPath == "fractionCompleted" {
            if zipProgress?.fractionCompleted == 1 {
                print("***ZIP COMPLETE***")
                removeZipFile()
                zipProgress?.removeObserver(self, forKeyPath: "fractionCompleted")
                DispatchQueue.main.asyncAfter(deadline: .now() + 2.5) {
                    // json was corrupt perhaps zip was cleaning up so try delaying
                    self.loadScales()
                    NotificationCenter.default.post(name: .ScalesDidDownloadUpdate, object: self)
                }
            } else {
                print("f:\(zipProgress?.fractionCompleted) \(zipProgress?.completedUnitCount)")
            }
        } else {
            super.observeValue(forKeyPath: keyPath, of: object, change: change, context: context)
        }
    }

ZIP Foundation processes everything synchronously.

Yes  I'm extracting asynchronously on a queue.

The progress count is incremented after ZIP Foundation has processed the entry content but before the destination file gets closed. (Therefore it's possible that file contents haven't been flushed to disk yet).
Perhaps it's better to use the progress for user feedback (e.g for progress bars) but not for driving your extraction logic.
An alternative approach to KVOing fractionCompleted would be to notify the next component of your extraction code when archive.extract(...) returns. At this point it is guaranteed that the destination file has been written and closed.

Perfect solution, why didn't I think of that?

Glad I could help.


Add end combinator to support matching end of routes.

This should solve issue #30

 Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.

Nice! Thank you :+1:


Add API to determine if user is a hub user

We need some method like user.isHubUser() that would allow us to determine if this is a local user or a hub user. In case, it is an admin, it should return false.

Now, on Subutai we divide Local user with Hub user only with email part.
Hub user has<EMAIL_ADDRESS>email format. Every time, I wondered if Subutai admin creates a user with same email format. So, this user will be counted as Hub user in the current implementation.

Afaik hub user for admin is not created

Afaik hub user for admin is not created
Admin can see and manage it on User Management page

Whenever a hub user creates an environment, its delegate user with an email containing @hub.subut.ai is created. For peer owner, no such delegate user is created if he tries to create an environment from Hub on his own peer.

I mean, peer owner (admin) can create a user on Subutai and can set any email to this user, this scenario is without Hub.

Whenever Hub user creates env in this case we better add new fields to user record in db whichis set only by system.


Library artifacts are published to jcenter only, which will close in 3 months

Hello! Your library is published only to jcenter. Please consider publishing it also to mavenCentral, as jcenter is shutting down in May.
Here is some info on how to publish artifacts to mavenCentral.

Latest release is now in mavenCentral: https://s01.oss.sonatype.org/content/groups/public/com/github/rubensousa/gravitysnaphelper/2.2.1/


Proposal to add an out of the box support of some premium pricing data set from quandl

Dear Zipline Maintainers,
There is an open problem with WIKI prices dataset. The thing is that WIKI prices is not being updated for several days, the last pricing data there is for 28th of February, 2018.
I emailed the quandl's support and here is their answer:
"Thanks for bringing this to our attention.
Your email has been forwarded to our data team so that this can be added to their development queue.
Please kindly note that as a free database, there is no guaranteed timeline by when our team will be able to investigate this issue. And, there is no guaranteed timeline by when they will be able to apply updates or corrections to the database.
If you need more reliable data, we recommend the following premium databases:
EOD: https://www.quandl.com/data/EOD
SEP: https://www.quandl.com/databases/SEP"
The presence of recent pricing data is not that important when backtesting strategies,
but very essential when trading live. I think  most of zipline-live users would consider to use payed data for live trading, so I propose to include a loader for one of the quandl's premium databases as an option, to make zipline-live to support reliable data out of the box
Sincerely,
Vitaly

One can take as basis the data ingestor for the ZEP quandl dataset:
https://github.com/pbharrin/alpha-compiler/blob/master/alphacompiler/data/loaders/zacks_quandl.py
The SEP prices dataset is not only cheaper but also compatible with the SF1 fundamentals dataset from the same data vendor.

I've just seen that @pbharrin implemented three days ago a loader for the SEP dataset:
https://github.com/pbharrin/alpha-compiler/blob/master/alphacompiler/data/loaders/sep_quandl.py
Very good, but there are some issues with the data as he pointed in his blog post:
http://alphacompiler.com/blog/9/


Vocab Files Validation

Might someone please run the reference files through a validator or linter ?
Thank you !!

Hm. Just something I ran into during implementation (and all of my parsers refused to read it…):
There appears to be a somewhat unhappy asterisk e.g. in line 14090 of the JSON.
Since I cannot find the respective term in the PDF version of the specification, I’m not completely sure, what the actual intention is here; probably more to create something like a list ?  Or simple set of value-options as in https://www.w3.org/2001/sw/BestPractices/OEP/Lists-of-values-20040625/ ?
Do you want gs1:defaultLink to be provided as soon as more than one instance ocurrs ?
At least I am not familiar with any asterisk-style syntax for this task… – or am I out-of-date ?

The problem seems to arise from the fact, that while a "*" is indeed perfectly valid JSON, it becomes an issue as soon as you use it in a Linked Data context where the used keys are resolving into IRIs.
And according to HTTP the "*" then is a reserved character which always needs to be escaped.

Indeed. @mgh128 has been screaming in my general direction on this one. I suspect we may need to rename that term in the coming months (perhaps literally to 'defaultLinkStar'). The reason I picked it in the first place is that it's a convention used in other places, that the * suffix on the end of foo means something like 'a richer version of foo.' (RDF* for instance!) But, OK, it was not a good idea here.

Not exactly screaming, but politely questioning whether it was a good idea.  I understand that it's inherited from the use of a trailing asterisk as proposed in https://tools.ietf.org/html/rfc8187, which appears also in https://datatracker.ietf.org/doc/draft-ietf-httpapi-linkset/?include_text=1  ( the Linkset draft by Erik Wilde and Herbert Van de Sompel  ).
However, in a world of Linked Data, the use of a restricted URI character such as asterisk is a very bad idea, because it needs percent escaping.  We should nowadays consider that for any attribute or property appearing in any data sharing standard, somebody is likely to want to construct a Web URI for it (if the original authors neglected to do so in the first place) - and therefore, it's better if the names of all attributes and properties only use safe URI characters, 0-9, A-Z a-z, hyphen and underscore.  Use of any other character is just asking for trouble.
If we really wanted to fix this at the root cause, then perhaps one of us should have the courage to submit a polite constructive comment in response to RFC 8187 and point out that asterisk is problematic and that title* etc. should be renamed titleMulti etc., in which the * is replaced by something entirely alphabetic, unproblematic and more meaningful.  At the same time, we should also provide this feedback to Erik and Herbert that asterisk is problematic and encourage them to adjust their Linkset draft before it reaches RFC status.

Following this discussion:

We have defined gs1:defaultLinkMulti to replace gs1:defaultLink*
Turtle and JSON-LD files have been updated
This is being treated as an erratum in the underlying spec that is going through its final stages within the GS1 standardization process and so defaultLink* is unlikely ever to see the light of day again, except as a note in the vocabulary here with the status of 'withdrawn'.

@bardonelgen where is the PDF version of the specification that you mentioned?

The PDF can be found at https://www.gs1.org/docs/gs1-smartsearch/GS1_Vocabulary_Standard.pdf — though, be cautious, this appears to be rather inconsistent and outdated compare to what you find here on GitHub…


registration process

I assume it is about registering new WIS2 Nodes ? If that is the case, this will be part of the guide.

20231013_Adding_NC_WIS2.pdf
20231013_Adding_WIS2_Node.pdf


[Suggestion] Add sudoers insults

You could add the insults from sudoers. They're in sudo/plugins/sudoers/. Here they are in a targz:
https://filesha.re/sudoers.tar.gz

Hi
I am not comfortable with just taking the sudo insults, and I also think it is fun with different insults on a system where both are activated.
However, to make it easy for you to include your own insults/messages without having to change the source code I added a couple of environment variables that you can configure in the form of CMD_NOT_FOUND_MSGS and CMD_NOT_FOUND_MSGS_APPEND as documented in the README.


docs(changelog) add 2.1.1 changes

Summary
i am waiting
Full changelog
Issues resolved

@basheerali 2.1.1 is coming up in the next few days! Closing this for now.


Close/Free for Performance?

Hello david. I loved the code.
First of all, it's worth clarifying that I'm a newbie in PHP.
I have tried some times to use PHP in previous projects (in shared hostings) and they have caught my attention because the scripts consumed a lot of memory, I remember that something I did to solve it was to close the queries and connections.
You do not need any of that in this wrapper?
I would appreciate it if you could clarify it to me.
Greetings.

Hi Ezetojo, thanks!
PDO closes the connection automatically at the end of the script so it shouldn't need manually closing, that is why I never put a closing part into the wrapper.
This is only a wrapper for PDO so if a lot of memory is being used it could be due to a complex query being executed.
You can add index to your database to make querying faster and also optimise your query by only selecting what you need. That should help use less memory.

Wow, thanks for the speed of response David!
Excellent, I suppose then that the previous times I was using mysqli.


doc.querySelector(".social-count")?.textContent error

Hi, i have some problem.
Can you help me please?
Argument of type 'string | null | undefined' is not assignable to parameter of type 'string'.
Type 'undefined' is not assignable to type 'string'.

Sorry, its not the same problem, as you can see at the picture debugger throw me only one error.
I make all that tell Asim, https://github.com/hieunc229/copilot-clone/issues/48#issuecomment-997348741
But it not help
Can you help me
How can i fix that
Thx

Today i see seond error message

File extension.ts not found
Very strange

Ok. Thx. I`ll be waiting. Good luck! :)

The error showing is the TypeError (not the same error message, but its complaints about the type definition). You can ignore it (since Captain Stack is using a new API, the dev team changes its type definition quite often).
Though, I dig a little more. Turn out it also made some changes recently. After installing Captain Stack as our instruction, you can follow this answer for the recent fix. https://github.com/microsoft/vscode/issues/124024#issuecomment-985328059
I'll make some changes based on this update

It's a type-safe error for older versions of Typescript, due to the use of optional chaining .? that could return undefined.
Bypassed it by making sure parseInt will always take a string:
const socialCount : string = doc.querySelector(".social-count")?.textContent?.toString() || ""

        if (!snippet) return [];

        const item: SnippetResult = {
            votes: parseInt(socialCount),
            code: cleanContent(snippet),
            sourceURL: options.url,
            hasCheckMark: false
        };


Connect to postgresql on specific port

Is there any way to connect to specific postgresql port ?

You can either specify the environment variable PGPORT, or use the patch from #340 to have wal-e automatically detect the proper port from PGDATA.


[BACKLOG-13032] groupId change for elastic & transitive fix in assembly

@e-cuellar @lgrill-pentaho

Build Skipped
:white_check_mark: There was nothing to build. The changes are probably excluded by the build rules. If this is an error, please contact your system administrator.


FATAL: Ident authentication failed for user

Hi all,
got this error setting up the database on linux
[opencue@opencue tmp]$ flyway -url=jdbc:postgresql://$DB_HOST/$DB_NAME -user=$USER -n -locations=filesystem:cuebot/src/main/resources/conf/ddl/postgres/migrations migrate
ERROR: Unable to obtain connection from database (jdbc:postgresql://localhost/cuebot_local) for user 'opencue': FATAL: Ident authentication failed for user "opencue"
SQL State  : 28000
Error Code : 0
Message    : FATAL: Ident authentication failed for user "opencue"
Caused by: org.postgresql.util.PSQLException: FATAL: Ident authentication failed for user "opencue"
[opencue@opencue tmp]$
I've created a superuser in postgres called opencue so I have no idea why it's not working
Any help would be much appreciated
thanks,
rob

ok what i did was
vi /var/lib/pgsql/data/pg_hba.conf
change all to "trust"
systemctl restart postgresql.service
now i get this
[opencue@opencue tmp]$ flyway -url=jdbc:postgresql://$DB_HOST/$DB_NAME -user=$USER -n -locations=filesystem:cuebot/src/main/resources/conf/ddl/postgres/migrations migrate
A new version of Flyway is available
Upgrade to Flyway 9.20.1: https://rd.gt/2X0gakb
Flyway Community Edition 9.8.1 by Redgate
See what's new here: https://flywaydb.org/documentation/learnmore/releaseNotes#9.8.1
ERROR: Skipping filesystem location: cuebot/src/main/resources/conf/ddl/postgres/migrations (not found)
Database: jdbc:postgresql://localhost/cuebot_local (PostgreSQL 13.11)
Successfully validated 0 migrations (execution time 00:00.009s)
WARNING: No migrations found. Are your locations set up correctly?
Current version of schema "public": << Empty Schema >>
Schema "public" is up to date. No migration necessary.
[opencue@opencue tmp]$

Hi @robina80 , could you check to ensure the location: cuebot/src/main/resources/conf/ddl/postgres/migrations  is existing?

ok you know what, i think im being dumb
looking at the database how-to
https://www.opencue.io/docs/getting-started/setting-up-the-database/
populate the database, im doing option 1 so im guessing i dont need to do
option 2 as well ?
On Wed, 19 Jul 2023 at 14:32, JasonLuo-Redgate @.***>
wrote:

Hi @robina80 https://github.com/robina80 , could you check to ensure
the location: cuebot/src/main/resources/conf/ddl/postgres/migrations is
existing?
—
Reply to this email directly, view it on GitHub
https://github.com/flyway/flyway/issues/3712#issuecomment-1642093302,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AD7SQJIAQID5MS6N63LPAYDXQ7OYBANCNFSM6AAAAAA2PAITLA
.
You are receiving this because you were mentioned.Message ID:
@.***>

--
Regards,
Robert K Wild.


Add clientId to survey results

It would be nice to see the clientId in the survey results.

ClientId is saved in post results. I'll check why it doesn't visible as a result column.

same here... i would like to see the ClientId as a value in the results, when i process the data i cant access this value.

I've added the clientid column to the survey results and updated sites.

This still does not allow for the clientId to be passed in the REST API call to https://surveyjs.io/api/MySurveys/getSurveyResults.

@tsv2013 Could you please check it out?
Thank you,
Andrew

This is the page with the results of my survey

As you can see client IP address is accessible.
But first you should turn this option on.
Another reason - your result columns can be customized and you need to show corresponding column.
And by default client ID column doesn't visible in public results link, You can change it via Customize Columns button on the results page.
If this doesn't help we'll need to check your certain survey.

@mnhospitals @andrewtelnov I'll check the API right now

@mnhospitals @andrewtelnov
The IPAddress column is present in the API results as well:


Traceback: Function pg_stat_get_wal_receiver() is currently not supported in Aurora

Using pg_activity 3.0.0a1 (commit<PHONE_NUMBER>58b606efd0c1d06166be0bec534ce9) against an AWS RDS/Aurora database, the following traceback occurs with psycopg2-binary==2.9.3
$ pg_activity -U <username> -h <host> -d <db> --rds --no-db-size

Traceback (most recent call last):
  File "/home/ubuntu/run2/bin/pg_activity", line 11, in <module>
    load_entry_point('pg-activity==3.0.0a1', 'console_scripts', 'pg_activity')()
  File "/home/ubuntu/run2/lib/python3.8/site-packages/pg_activity-3.0.0a1-py3.8.egg/pgactivity/cli.py", line 347, in main
    ui.main(term, dataobj, host, args)
  File "/home/ubuntu/run2/lib/python3.8/site-packages/pg_activity-3.0.0a1-py3.8.egg/pgactivity/ui.py", line 29, in main
    server_information = data.pg_get_server_information(
  File "/home/ubuntu/run2/lib/python3.8/site-packages/pg_activity-3.0.0a1-py3.8.egg/pgactivity/data.py", line 225, in pg_get_server_information
    cur.execute(
  File "/home/ubuntu/run2/lib/python3.8/site-packages/psycopg2/extras.py", line 146, in execute
    return super().execute(query, vars)
psycopg2.errors.FeatureNotSupported: Function pg_stat_get_wal_receiver() is currently not supported in Aurora

Thanks for the report, will look into it.

Hi,
Master should fix your problem. It disables wal receiver stats in case of failure or if you set "--no-walreceiver".
Feel free to reopen, in case it's still doesn't work.


feat: add SAS content pane

Summary
This introduces the first iteration of SAS content navigator. In this first iteration, the following functionality is introduced:

File and folder creation
File, folder, and favorites renaming
File and folder deletion

One thing to call out here is that File deletion is permanent in this iteration. In a future iteration, we'll attempt to bring over the recycle bin functionality currently present in SAS drive.
Testing

[x] Test creating a file*
[x] Test creating a folder*
[x] Test renaming a file*
[x] Test deleting a file*
[x] Test renaming a file in My Favorites* (renaming a file or folder renames the link, but not the actual underlying resource. However, renaming a file w/n a link folder does update the file in both places)
[x] Test signing in (both with and without profile)
[x] Test opening a variety of files (sas programs but also json files / etc)
[x] Test refresh functionality
[x] Test collapse all functionality
[x] Check with @smorrisj about how this functionality is impacted by v9 (where SAS content is unsupported)
[ ] Make sure messages are finalized with @skyeaussie and @crlewi before merging

For each test case, test in root folder and sub folder. Test that file actions with naming collisions (i.e. creating a file with a name that already exists) fails

Hi @scottdover @boyce-w,
When working with the compute service, we decided to make service call via OpenAPI generated code. It ensures we follow the API spec and help developers with inline assist with TypeScript’s type system. Developers don't need to check API and manually write service calls.
I generated API files for folders and files services and attached to https://github.com/sassoftware/vscode-sas-extension/pull/112
Do you think it can benefit?

Hi @scottdover @boyce-w, When working with the compute service, we decided to make service call via OpenAPI generated code. It ensures we follow the API spec and help developers with inline assist with TypeScript’s type system. Developers don't need to check API and manually write service calls. I generated API files for folders and files services and attached to #112 Do you think it can benefit?

@scnwwu I think incorporating this is likely a good idea. That said, I'm wondering if we can tackle that in https://github.com/sassoftware/vscode-sas-extension/issues/109, since this could be a larger change. Does that sound acceptable?

Hi @scottdover @boyce-w, When working with the compute service, we decided to make service call via OpenAPI generated code. It ensures we follow the API spec and help developers with inline assist with TypeScript’s type system. Developers don't need to check API and manually write service calls. I generated API files for folders and files services and attached to #112 Do you think it can benefit?

@scnwwu I think incorporating this is likely a good idea. That said, I'm wondering if we can tackle that in #109, since this could be a larger change. Does that sound acceptable?

Good to me

@scnwwu I briefly looked into using https://github.com/microsoft/vscode-extension-samples/tree/main/l10n-sample for localization purposes (and having a common place to store strings). That said, this API is very new and doesn't seem to be working as expected. I think it's potentially something to look into in the future, though. It'd be nice to have all of our messaging in one place for ux to review.

@scnwwu I briefly looked into using https://github.com/microsoft/vscode-extension-samples/tree/main/l10n-sample for localization purposes (and having a common place to store strings). That said, this API is very new and doesn't seem to be working as expected. I think it's potentially something to look into in the future, though. It'd be nice to have all of our messaging in one place for ux to review.
UPDATE: Looks like having this issue resolved would put us in a better spot - microsoft/vscode#168127. I'll hold off on these changes until progress is made there

Yeah, l10n should definitely be a separate task. My comment only meant for the package.nls.json file so that we have a good foundation.

Hey @Becky-Williamson. Can you confirm the issue with deleting files/favorites is resolved?

Added issue https://github.com/sassoftware/vscode-sas-extension/issues/130

Hey @smorrisj @scnwwu @boyce-w! Can I get one more quick look over this before merging?


Add Win_x86/Win_X64 builds and improve JS code

I generally find underscore code much cleaner than raw javascript -- and performance is never a consideration on the web wasteland of cpu cycles. Any reason you don't like it?

I must say the end result looks BEAUTIFUL though :D


SteamVR teleport button sends me to a new scene

I have two scenes in my build and when I load scene one I have to teleport to wake up the controllers and move around the scene. However as soon as I press the track pad button the second scene loads.
It should only be loading the scene when the hand touches the exit point.
    public string TriggerObjName; // Set to a collider trigger on the steamVR Hands

    private void OnTriggerEnter(Collider other)
    {
        if (other.gameObject.name.ToLower().IndexOf(TriggerObjName.ToLower()) > -1)
        {
            SceneManager.LoadScene("Scene2");
        }

And it doesn't matter if that script is on or not. The script can be removed from the project and the teleport button still sends me to scene two.

Never mind. I fond the Steam VR_Load Level script


Save Spanish Chars?

is a way to save this kind of chars -> ç¡'ºª\¨·¿ñ
as their original form?

Set your db to use utf8 when you create it. There is no special support in knex to make it happen. Also knex usage questions should not be done here as issue template also mentions.


Extend locking mechanism to different resources

Is your idea related to a problem? Please describe.
We implemented locking for dataset level. However, we are now facing issues when 2 or more shares are getting approved for the same requestor at the same time. This gives rise to a race condition where the IAM policy of requestor is modified at the same time.
We observed an error like:
Failed to update policy SHARE_POLICY_ARN : An error occurred (NoSuchEntity) when calling the DeletePolicyVersion operation: Policy SHARE_POLICY_ARN version v3 does not exist or is not attachable.

Example:

UserA in GroupA requests shares to bucket in Dataset B_1 and folder in Dataset B_2
GroupB (who owns both datasets) approves the share at the same time
Both shares will be processed at the same time and update the IAM share policy of UserA to give S3 and KMS permissions to the resources (here causes the error)

Describe the solution you'd like
Think about a way to extend locking mechanism beyond just dataset locking to different resource types. What if we need to lock resource X during activity Y (i.e. lock datasets from other actions when updating, or similar)
P.S. Don't attach files. Please, prefer add code snippets directly in the message body.

@anushka-singh thanks for opening an issue. This feature came up during validation of 2.3. Do you think you have capacity to take on the extension? In any case, we have added it to our 2.5 project and try to pick it up as soon as possible.

@dlpzx I will be working on expiration of shares feature next. But if I have bandwidth for 2.5 then can pick this up shortly after.

Thoughts on the design for this issue below...
Changes to DB Schema

Rename dataset_lock to resource_lock
Rename column datasetUri to resourceUri
Add column acquiredByResourceType to represent a string/enum value for what resource it is acquired by (i.e. share, etc.)

This will be additional information alongside the acquiredBy Column to store the shareUri
Do we think this would be useful in long-run (i.e. think if we extend locking to not only be a part of the sharing mechanism) ?

Changes to Share Logic

Check if either datasetUri is locked OR Requester Principal is locked

OPEN ENDED Q: The primary key for Env Group is composite key of groupUri and environmentUri

This issue is that GroupA can request access from many dif Envs (need to check lock for groupUri-envUri pair!)
Option A - in resourceUri column store concatenated Uri Pair ('groupUri-envUri')
Option B - in resourceUri column store environmentIAMRoleArn (in this case maybe resourceUri is the wrong name for the column)
Option C - introduce resourceEnvUri column and make it a composite key along with resourceUri ++ backfilling

Please take a look @anushka-singh @dlpzx - I will get on implementing the above shortly but there are some open questions I would appreciate input on

Hey Noah,
Thanks for picking this up and coming up with a design on this.
I am OOO tomorrow, but I will get back to you on this by Thursday.

Thoughts on the design for this issue below...
Changes to DB Schema

Rename dataset_lock to resource_lock

Rename column datasetUri to resourceUri

Add column acquiredByResourceType to represent a string/enum value for what resource it is acquired by (i.e. share, etc.)

This will be additional information alongside the acquiredBy Column to store the shareUri
Do we think this would be useful in long-run (i.e. think if we extend locking to not only be a part of the sharing mechanism) ?

Agree to the DB schema changes.
Yes I think we should extend locking and not just keep it restricted to sharing. We have discussed in the past that we should introduce resource level locking for datasets, environments and organizations etc when they are undergoing updates. It can be used in tandem with the maintenance window work to ensure processes are paused when updates on these resources start and are picked up again when maintenance window is complete.

Changes to Share Logic

Check if either datasetUri is locked OR Requester Principal is locked

OPEN ENDED Q: The primary key for Env Group is composite key of groupUri and environmentUri

This issue is that GroupA can request access from many dif Envs (need to check lock for groupUri-envUri pair!)
Option A - in resourceUri column store concatenated Uri Pair ('groupUri-envUri')
Option B - in resourceUri column store environmentIAMRoleArn (in this case maybe resourceUri is the wrong name for the column)
Option C - introduce resourceEnvUri column and make it a composite key along with resourceUri ++ backfilling

For checking  env group, I like option A the best. It is easy to implement and does not need a new column introduced either.

Please take a look @anushka-singh @dlpzx - I will get on implementing the above shortly but there are some open questions I would appreciate input on

We have implemented this extension of locking mechanism to be generic for any resource rather than dataset specific in PR #1338
We went with Option A above as a rule of thumb to concatenate Uri Pair for records that have composite primary keys

For example for EnvGroups (table = enviornment_group_permission) the composite key is groupUri and environmentUri so the primary key identifier in Resource Lock table is 'groupUri-envUri'

We also changed the model such that acquire_lock will create records in the table and release_lock will delete records - this way there is no requirement to add create_lock and delete_lock logic whenever we create/delete a new resource and any resource can leverage the locking class functions without additional custom logic
I will go ahead and close this issue - thanks for all the design considerations and help in above thread!
cc: @anushka-singh


Unable to use local models with ollama

I am running a local llama server with ./ollama serve, which runs on port 11434. Then, in another shell, I have a model running with  ./ollama run llama2-uncensored. Here is what my .env.local looks like
# Supabase Public
NEXT_PUBLIC_SUPABASE_URL=http://<IP_ADDRESS>:54321
NEXT_PUBLIC_SUPABASE_ANON_KEY=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJzdXBhYmFzZS1kZW1vIiwicm9sZSI6ImFub24iLCJleHAiOjE5ODM4MTI5OTZ9.CRXP1A7WOeoJeXxj>
# Supabase Private
SUPABASE_SERVICE_ROLE_KEY=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJzdXBhYmFzZS1kZW1vIiwicm9sZSI6InNlcnZpY2Vfcm9sZSIsImV4cCI6MTk4MzgxMjk5Nn0.EGIM96RAZ>
# Ollama
NEXT_PUBLIC_OLLAMA_URL=http://localhost:11434
# API Keys (Optional: Entering an API key here overrides the API keys globally for all users)
OPENAI_API_KEY=
ANTHROPIC_API_KEY=
GOOGLE_GEMINI_API_KEY=
MISTRAL_API_KEY=
PERPLEXITY_API_KEY=
OPENROUTER_API_KEY=

# OpenAI API Information
NEXT_PUBLIC_OPENAI_ORGANIZATION_ID=

# Azure API Information
AZURE_OPENAI_API_KEY=
NEXT_PUBLIC_AZURE_OPENAI_ENDPOINT=
NEXT_PUBLIC_AZURE_GPT_35_TURBO_ID=
NEXT_PUBLIC_AZURE_GPT_45_VISION_ID=
NEXT_PUBLIC_AZURE_GPT_45_TURBO_ID=
#Extras
REACT_EDITOR=code

When I run npm run chat I get the following error
(node:78610) [DEP0040] DeprecationWarning: The `punycode` module is deprecated. Please use a userland alternative instead.
(Use `node --trace-deprecation ...` to show where the warning was created)

Now the ui loads correctly but I am unable to send messages/receive responses
The following error appears in the ui
Unhandled Runtime Error
Error: TypeError: Failed to fetch

  10 |
  11 | if (!profile) {
> 12 |   throw new Error(error.message)
     |        ^
  13 | }
  14 |
  15 | return profile

Call Stack
async fetchStartingData
components/utility/global-state.tsx (170:22)

Please help

warining is not an error, and you don't need to run a model for it to be usable from the ollama api. and your last error seems to be an account problem, not an ollama one.

http://localhost:54323/project/default/auth/users look here if u see an user

@wrapss I restated everything without running a model still no luck and I checked http://localhost:54323/project/default/auth/users there is a user listed with the email i used to test
still unable to send any message I can type into the box but cannot send

I also tested the ollama api to make sure its working with
curl http://localhost:11434/api/generate -d '{
  "model": "llama2-uncensored",
  "prompt":"Why is the sky blue?"
}'

It works as intended

you get the error related to the profile when you type a message? please be precise

Do you see the profile icon in the bottom left corner?

what version of ollama do you have? v0.1.14 is required
try this request for check :
curl http://localhost:11434/api/chat -d '{
"model": "mistral",
"messages": [
{ "role": "system", "content": "You are a helpful assistant that answers concisely." },
{ "role": "user", "content": "why is the sky blue?" }
]
}'

you get the error related to the profile when you type a message? please be precise
No the error pops up on the bottom left corner as soon as the UI loads

Do you see the profile icon in the bottom left corner?
No I do not

what version of ollama do you have? v0.1.14 is required try this request for check : curl http://localhost:11434/api/chat -d '{ "model": "llama2-uncensored", "messages": [ { "role": "system", "content": "You are a helpful assistant that answers concisely." }, { "role": "user", "content": "why is the sky blue?" } ] }'

Works perfectly repose below-
{"model":"llama2-uncensored","created_at":"2024-01-21T19:13:28.914668982Z","message":{"role":"assistant","content":"The"},"done":false}
{"model":"llama2-uncensored","created_at":"2024-01-21T19:13:29.105050915Z","message":{"role":"assistant","content":" sky"},"done":false}
{"model":"llama2-uncensored","created_at":"2024-01-21T19:13:29.288647982Z","message":{"role":"assistant","content":" appears"},"done":false}
{"model":"llama2-uncensored","created_at":"2024-01-21T19:13:29.497007569Z","message":{"role":"assistant","content":" blue"},"done":false}
{"model":"llama2-uncensored","created_at":"2024-01-21T19:13:29.675832943Z","message":{"role":"assistant","content":" because"},"done":false}
{"model":"llama2-uncensored","created_at":"2024-01-21T19:13:29.883213972Z","message":{"role":"assistant","content":" the"},"done":false}
{"model":"llama2-uncensored","created_at":"2024-01-21T19:13:30.037603269Z","message":{"role":"assistant","content":" air"},"done":false}
{"model":"llama2-uncensored","created_at":"2024-01-21T19:13:30.20696632Z","message":{"role":"assistant","content":" mole"},"done":false}
{"model":"llama2-uncensored","created_at":"2024-01-21T19:13:30.403519511Z","message":{"role":"assistant","content":"cules"},"done":false}
{"model":"llama2-uncensored","created_at":"2024-01-21T19:13:30.622470602Z","message":{"role":"assistant","content":" in"},"done":false}
{"model":"llama2-uncensored","created_at":"2024-01-21T19:13:30.82784807Z","message":{"role":"assistant","content":" Earth"},"done":false}
{"model":"llama2-uncensored","created_at":"2024-01-21T19:13:31.014603151Z","message":{"role":"assistant","content":"'"},"done":false}
{"model":"llama2-uncensored","created_at":"2024-01-21T19:13:31.199900989Z","message":{"role":"assistant","content":"s"},"done":false}
{"model":"llama2-uncensored","created_at":"2024-01-21T19:13:31.400349694Z","message":{"role":"assistant","content":" atmosphere"},"done":false}
{"model":"llama2-uncensored","created_at":"2024-01-21T19:13:31.554062794Z","message":{"role":"assistant","content":" scatter"},"done":false}
{"model":"llama2-uncensored","created_at":"2024-01-21T19:13:31.734748944Z","message":{"role":"assistant","content":" the"},"done":false}
{"model":"llama2-uncensored","created_at":"2024-01-21T19:13:31.914480821Z","message":{"role":"assistant","content":" sun"},"done":false}
{"model":"llama2-uncensored","created_at":"2024-01-21T19:13:32.103455475Z","message":{"role":"assistant","content":"light"},"done":false}
{"model":"llama2-uncensored","created_at":"2024-01-21T19:13:32.30884549Z","message":{"role":"assistant","content":" that"},"done":false}
{"model":"llama2-uncensored","created_at":"2024-01-21T19:13:32.503019283Z","message":{"role":"assistant","content":" reaches"},"done":false}
{"model":"llama2-uncensored","created_at":"2024-01-21T19:13:32.703166746Z","message":{"role":"assistant","content":" them"},"done":false}
{"model":"llama2-uncensored","created_at":"2024-01-21T19:13:32.869124364Z","message":{"role":"assistant","content":","},"done":false}
{"model":"llama2-uncensored","created_at":"2024-01-21T19:13:33.075085867Z","message":{"role":"assistant","content":" allowing"},"done":false}
{"model":"llama2-uncensored","created_at":"2024-01-21T19:13:33.259654233Z","message":{"role":"assistant","content":" shorter"},"done":false}
{"model":"llama2-uncensored","created_at":"2024-01-21T19:13:33.484047904Z","message":{"role":"assistant","content":" w"},"done":false}
{"model":"llama2-uncensored","created_at":"2024-01-21T19:13:33.651021459Z","message":{"role":"assistant","content":"avel"},"done":false}
{"model":"llama2-uncensored","created_at":"2024-01-21T19:13:33.819481813Z","message":{"role":"assistant","content":"ength"},"done":false}
{"model":"llama2-uncensored","created_at":"2024-01-21T19:13:33.989786979Z","message":{"role":"assistant","content":" colors"},"done":false}
{"model":"llama2-uncensored","created_at":"2024-01-21T19:13:34.176107756Z","message":{"role":"assistant","content":" such"},"done":false}
{"model":"llama2-uncensored","created_at":"2024-01-21T19:13:34.351725334Z","message":{"role":"assistant","content":" as"},"done":false}
{"model":"llama2-uncensored","created_at":"2024-01-21T19:13:34.545319027Z","message":{"role":"assistant","content":" blue"},"done":false}
{"model":"llama2-uncensored","created_at":"2024-01-21T19:13:34.724989293Z","message":{"role":"assistant","content":" to"},"done":false}
{"model":"llama2-uncensored","created_at":"2024-01-21T19:13:34.909247648Z","message":{"role":"assistant","content":" pass"},"done":false}
{"model":"llama2-uncensored","created_at":"2024-01-21T19:13:35.093301414Z","message":{"role":"assistant","content":" through"},"done":false}
{"model":"llama2-uncensored","created_at":"2024-01-21T19:13:35.270426292Z","message":{"role":"assistant","content":" more"},"done":false}
{"model":"llama2-uncensored","created_at":"2024-01-21T19:13:35.484654243Z","message":{"role":"assistant","content":" easily"},"done":false}
{"model":"llama2-uncensored","created_at":"2024-01-21T19:13:35.640871752Z","message":{"role":"assistant","content":" than"},"done":false}
{"model":"llama2-uncensored","created_at":"2024-01-21T19:13:35.848182878Z","message":{"role":"assistant","content":" longer"},"done":false}
{"model":"llama2-uncensored","created_at":"2024-01-21T19:13:35.990402542Z","message":{"role":"assistant","content":" w"},"done":false}
{"model":"llama2-uncensored","created_at":"2024-01-21T19:13:36.151532042Z","message":{"role":"assistant","content":"avel"},"done":false}
{"model":"llama2-uncensored","created_at":"2024-01-21T19:13:36.320312113Z","message":{"role":"assistant","content":"ength"},"done":false}
{"model":"llama2-uncensored","created_at":"2024-01-21T19:13:36.502042065Z","message":{"role":"assistant","content":" colors"},"done":false}
{"model":"llama2-uncensored","created_at":"2024-01-21T19:13:36.695123002Z","message":{"role":"assistant","content":" like"},"done":false}
{"model":"llama2-uncensored","created_at":"2024-01-21T19:13:36.910703114Z","message":{"role":"assistant","content":" red"},"done":false}
{"model":"llama2-uncensored","created_at":"2024-01-21T19:13:37.11855139Z","message":{"role":"assistant","content":"."},"done":false}
{"model":"llama2-uncensored","created_at":"2024-01-21T19:13:37.310841441Z","message":{"role":"assistant","content":""},"done":true,"total_duration":19644627402,"load_duration":8208578028,"prompt_eval_count":38,"prompt_eval_duration":3039127000,"eval_count":46,"eval_duration":8396233000}

do you see the profile setting icon on the bottom left?

do you see the profile setting icon on the bottom left?

No I do not
Heres a screenshot of the page as it appears for me

so it's an account problem, try clearing chatbot-ui's cookies and try logging in again?

Im getting the same error however running on Caddy via CloudFlare tunnels

please try with the latest version

Same error - completely deleted everything and started from scratch. Unable to my profile context ("What would you like the AI to know about you to provide better responses?"), however supabase admin login is created.
`chatbot-ui % npm run chat

<EMAIL_ADDRESS>chat
supabase start && npm run db-types && npm run dev

supabase start is already running.
Run supabase status to show status of local Supabase containers.

<EMAIL_ADDRESS>db-types
supabase gen types typescript --local > supabase/types.ts

(node:1) ExperimentalWarning: Import assertions are not a stable feature of the JavaScript language. Avoid relying on their current behavior and syntax as those might change in a future version of Node.js.
(Use node --trace-warnings ... to show where the warning was created)
(node:1) ExperimentalWarning: Importing JSON modules is an experimental feature and might change at any time

<EMAIL_ADDRESS>dev
next dev

▲ Next.js 14.1.0

Local:        http://localhost:3000
Environments: .env.local

✓ Ready in 2.8s
✓ Compiled /middleware in 276ms (156 modules)
No valid locales in accept-language header: *
Reverting to using defaultLocale: en
○ Compiling /[locale] ...
✓ Compiled /[locale] in 8.8s (1593 modules)
No valid locales in accept-language header: *
Reverting to using defaultLocale: en
✓ Compiled in 934ms (692 modules)
✓ Compiled in 149ms (678 modules)
(node:21190) [DEP0040] DeprecationWarning: The punycode module is deprecated. Please use a userland alternative instead.
(Use node --trace-deprecation ... to show where the warning was created)
(node:21149) [DEP0040] DeprecationWarning: The punycode module is deprecated. Please use a userland alternative instead.
(Use node --trace-deprecation ... to show where the warning was created)
No valid locales in accept-language header: *
Reverting to using defaultLocale: `en``
Screenshots of error when trying to save profile context ("What would you like the AI to know about you to provide better responses?") and does not save (have tried clearing cookies, completely new reinstall of chatbotui):

try supabase db push ?

try supabase db push ?

I have a local install on a m1 MacBook Pro using Docker Desktop. How do I do the supabase db push given a default Chatbot-UI installation?

from the root of chatbot-ui folder, in the same way as you would use the supabase status command?

from the root of chatbot-ui folder, in the same way as you would use the supabase status command?

I see. Thank you. It seems like there is some intermediary step to be able to do this and that particular error seems to have been resolved by running "npm run update" from the chatbot root. None the less, this is the response I get when I run "supabase db push":
chatbot-ui % supabase db push Cannot find project ref.

How did you get the supabase credentials if you can't run supabase status?

in your .env you added NEXT_PUBLIC_SUPABASE_ANON_KEY and SUPABASE_SERVICE_ROLE_KEY? how did you get them?

in your .env you added NEXT_PUBLIC_SUPABASE_ANON_KEY and SUPABASE_SERVICE_ROLE_KEY? how did you get them?

supabase status works and provides me keys. Keys are correctly entered in env.local at ~/chatbot-ui/.
For whatever reason, the command supabase db push returns the error Cannot find project ref.
This is my first time working with supabase so I apologize that I cannot be more helpful.  Please let me know if there is anything else specifically that I can do that might be helpful. Doing my best to contribute how I can.

Same error - completely deleted everything and started from scratch. Unable to my profile context ("What would you like the AI to know about you to provide better responses?"), however supabase admin login is created.
chatbot-ui % npm run chat

><EMAIL_ADDRESS>chat
> supabase start && npm run db-types && npm run dev

supabase start is already running.
Run supabase status to show status of local Supabase containers.

><EMAIL_ADDRESS>db-types
> supabase gen types typescript --local > supabase/types.ts

(node:1) ExperimentalWarning: Import assertions are not a stable feature of the JavaScript language. Avoid relying on their current behavior and syntax as those might change in a future version of Node.js.
(Use `node --trace-warnings ...` to show where the warning was created)
(node:1) ExperimentalWarning: Importing JSON modules is an experimental feature and might change at any time

><EMAIL_ADDRESS>dev
> next dev

   ▲ Next.js 14.1.0
   - Local:        http://localhost:3000
   - Environments: .env.local

 ✓ Ready in 2.8s
 ✓ Compiled /middleware in 276ms (156 modules)
No valid locales in accept-language header: *
Reverting to using defaultLocale: en
 ○ Compiling /[locale] ...
 ✓ Compiled /[locale] in 8.8s (1593 modules)
No valid locales in accept-language header: *
Reverting to using defaultLocale: en
 ✓ Compiled in 934ms (692 modules)
 ✓ Compiled in 149ms (678 modules)

Screenshots of error when trying to save profile context ("What would you like the AI to know about you to provide better responses?") and does not save (have tried clearing cookies, completely new reinstall of chatbotui):

Ah exactly the errors I am getting. Thanks for the screenshot.
I tried clearing cookies and even a new deleting the repo and repluuing with a clean install of everything no luck

@mysterium-coniunctionis
Ah exactly the errors I am getting. Thanks for the screenshot. I tried clearing cookies and even a new deleting the repo and repluuing with a clean install of everything no luck

The update pushed yesterday fixed the profile context bug for me. I am now able to save my profile context via localhost:3000. I ran the update using npm run update.

@mysterium-coniunctionis I tried that now and I also tried redowloading the repo and starting from scratch I still get this error

However this is on http://localhost:3000/setup

I am pretty sure I have heres the .env.local
# Supabase Public
NEXT_PUBLIC_SUPABASE_URL=http://<IP_ADDRESS>:54321
NEXT_PUBLIC_SUPABASE_ANON_KEY=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJzdXBhYmFzZS1kZW1vIiwicm9sZSI6ImFub24iLCJleHAiOjE5ODM4MTI5OTZ9.CRXP1A7WOeoJeXxjNni43kdQwgnWNReilDMblYTn_I0

# Supabase Private
SUPABASE_SERVICE_ROLE_KEY=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJzdXBhYmFzZS1kZW1vIiwicm9sZSI6InNlcnZpY2Vfcm9sZSIsImV4cCI6MTk4MzgxMjk5Nn0.EGIM96RAZx35lJzdJsyH-qQwv8Hdp7fsn3W0YpN8>
# Ollama
NEXT_PUBLIC_OLLAMA_URL=http://localhost:11434
# API Keys (Optional: Entering an API key here overrides the API keys globally for all users)
OPENAI_API_KEY=
ANTHROPIC_API_KEY=
GOOGLE_GEMINI_API_KEY=
MISTRAL_API_KEY=
PERPLEXITY_API_KEY=
OPENROUTER_API_KEY=

# OpenAI API Information
NEXT_PUBLIC_OPENAI_ORGANIZATION_ID=

# Azure API Information
AZURE_OPENAI_API_KEY=
NEXT_PUBLIC_AZURE_OPENAI_ENDPOINT=
NEXT_PUBLIC_AZURE_GPT_35_TURBO_ID=
NEXT_PUBLIC_AZURE_GPT_45_VISION_ID=
NEXT_PUBLIC_AZURE_GPT_45_TURBO_ID=

and heres what supabase status returns
Stopped services: [supabase_analytics_chatbotui]
supabase local development setup is running.

        API URL: http://<IP_ADDRESS>:54321
    GraphQL URL: http://<IP_ADDRESS>:54321/graphql/v1
         DB URL: postgresql://postgres:postgres@<IP_ADDRESS>:54322/postgres
     Studio URL: http://<IP_ADDRESS>:54323
   Inbucket URL: http://<IP_ADDRESS>:54324
     JWT secret: super-secret-jwt-token-with-at-least-32-characters-long
       anon key: eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJzdXBhYmFzZS1kZW1vIiwicm9sZSI6ImFub24iLCJleHAiOjE5ODM4MTI5OTZ9.CRXP1A7WOeoJeXxjNni43kdQwgnWNReilDMblYTn_I0
service_role key: eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJzdXBhYmFzZS1kZW1vIiwicm9sZSI6InNlcnZpY2Vfcm9sZSIsImV4cCI6MTk4MzgxMjk5Nn0.EGIM96RAZx35lJzdJsyH-qQwv8Hdp7fsn3W0YpN81IU

Have I made some mistake

Tried a full update/reinstall again still had the same problem in creating a user

Call Stack
async fetchStartingData
components/utility/global-state.tsx (192:22)
async fetchData
components/utility/global-state.tsx (134:22)

You accidentally entered your role key incorrectly.

SUPABASE_SERVICE_ROLE_KEY=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJzdXBhYmFzZS1kZW1vIiwicm9sZSI6InNlcnZpY2Vfcm9sZSIsImV4cCI6MTk4MzgxMjk5Nn0.EGIM96RAZx35lJzdJsyH-qQwv8Hdp7fsn3W0YpN8>

This should be SUPABASE_SERVICE_ROLE_KEY=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJzdXBhYmFzZS1kZW1vIiwicm9sZSI6InNlcnZpY2Vfcm9sZSIsImV4cCI6MTk4MzgxMjk5Nn0.EGIM96RAZx35lJzdJsyH-qQwv8Hdp7fsn3W0YpN81IU`

@mckaywrigley I Just double-checked. Sorry, it was a mistake in copying my .env.local file
I think I have it correct I still have the same issue
Here is the proper set of environmental variables I am using I am still having the same
envvar.txt
I am still having the same issue please dont close the topic

I'm having exactly the same error, when accessing local it works fine, everything, but when accessing from a different computer then I have that error.
http://localhost:3000 -- works fine
http://<IP_ADDRESS>:3000 -- I get that error that the profile doesn't exists
I'm sure that .env.local it's right because in local works fine.

the ip of supabase must be the private ipv4 of your machine, not localhost or <IP_ADDRESS>

the ip of supabase is the ip of your machine, I don't understand the question?

Thank you @wrapss  , changing just the URL of SUPABASE_URL , the problem persists, changing both URLs (Supabase URL and Ollama URL) fixed the problem.
Now I can access the Chatbot UI from a different PC from my LAN.
Thanks!

Wonderful thanks that fixed my error!
Thank you so much!
Just one slight hiccup is that the chat output is significantly worse than the output I get from just using the ollama api

By the way, sorry for this new reply but I think that could be useful for someone using Chatbot UI in Windows with WSL and Ollama.
After being able to access using a different PC from my LAN, I noticed that local LLMs (LLama, etc.) were not showing up, the problem was changing the URL, that WSL needs to open some ports and route the traffic.
Following this steps fixed it and now I can use Local LLMs too: https://github.com/ollama/ollama/issues/1431


[swiftpm] For development, build a local copy of PackageDescription

When developing sourcekit-lsp, it is useful to be able to use a released
version of the swift toolchain. However, because we are using the latest
branch version of libSwiftPM, it may be that the PackageDescription
module packaged with the toolchain is incompatible with that libSwiftPM.
To prevent that problem, we now build our own copy of PackageDescription
and using that instead of the toolchain's. That change is made by
default to allow plain swift build and swift test invocations to
work without modification. When building as part of the swift toolchain,
we continue to use the toolchain's copy of PackageDescription, because
we know that it is built from the same sources as we are using for
libSwiftPM.
One situation that is not explicitly covered by the above is if
sourcekit-lsp is installed somewhere outside of a swift toolchain. In
that case by default it will try to find a local package description
module next to the sourcekit-lsp binary, but assuming that fails it will
fallback to searching the default toolcahin that it discovers at
runtime, as it does today. If this usecase turns out to be important we
can provide additional ways to control how to find the
PackageDescription module.
rdar://78548287

@swift-ci please test

Marking this a draft until I can test both development builds and and installed toolchain on both Linux and macOS.
I'd appreciate feedback on this approach, in particular

Are the module paths I'm checking for correct for all platforms (@compnerd will this work on Windows?)
I'm changing the default behaviour of swift build and swift test rather than making building PackageDescription opt-in. To me it's important that swift test is as simple as possible.  It will still fallback to the existing path if it cannot find PackageDescription relative to the sourcekit-lsp binary.
Is specifying binDir an acceptable use of the libSwiftPM API?  This is what swiftpm itself does during testing, but obviously what a project does for their own use is different from what they want to support.  If there was a better way to customize the PackageDescription, I'd be happy to adopt that instead.

I'll try to get a listing of the build tree.

This change has become obsolete.


Conditional requirements

Hi, I have tried to add an "if", "then", "else" in my schema, to add conditional required fields but I cannot make it work :-(
Using the validator I get a validation error (as expected) if I use the following:
"required": [           "transactionIdentifier",           "reasonCode",           "meteringpointId",           "contractEnd"         ]
but if I change this to:
"required": [           "transactionIdentifier",           "reasonCode",           "meteringpointId"         ],         "if": {"properties": {"header": {"properties": {"messageType": {"const": "406"}}}}},         "then": {"required":  ["contractEnd"]},         "else": {"required":  ["contractEnd"]}
I would expect the same result, since I have the same resulting "required" no matter what the condition..
But no, the field is not required...
So my question is, since "if" cases are not included in your Data folder. It is not implemented?

Hmm I guess I found the answer. this library is only draft-04 compliant right? Any chance of the library being updated to 07?

Hi. Indeed, this is a 04-only lib for now. I'd like to catch up with the spec at some point, but by lack of time, it probably won't happen in the immediate future.


Add a flag to RecordedTestBase to enable Fiddler usage with Test Proxy

Fiddler doesn't track localhost traffic. The workaround is to make the host "fiddler.localhost". We should add a flag that users can set to RecordedTestBase to make this easy when debugging locally.

/cc @seanmcc-msft


Added alacritty terminal colorscheme.

Love the colorscheme :heart: I usually dislike having my terminal colorscheme clash with my nvim colorscheme, so I made a very basic colorscheme for my terminal emulator of choice, alacritty. If you want me to update it or think some other colors may appear better, let me know.

Great, Thanks!


'approval-notes' is a documented input, but is missing in action.yaml

The README indicates that approval-notes is a supported input: https://github.com/wdzeng/firefox-addon#:~:text=approval-notes%3A (optional)%3A a secret text to Mozilla reviewers.
However, in the implementation this not so: https://github.com/wdzeng/firefox-addon/blob/main/action.yml#L5-L24
Was this dropped or missed in a recent update?

Confirmed that this appears to be all wired up in the code that makes the requests, so I sent a PR to add the input to the yaml


Missing frfuel.net.dll

Hello, there seems to be a file missing "frfuel.net.dll".
As stated in the __resource.lua below:
resource_manifest_version '44febabe-d386-4d18-afbe-5e627f4af937' files {      "config.ini",     "Newtonsoft.Json.dll",     "GasStations.json" } client_script "frfuel.net.dll"
I am unable to find "frfuel.net.dll".
Which causes an error upon trying to join the server.

Try reinstalling the resource, it appears something has deleted that particular file. Follow installation instructions precisely! If issue will remain please open new issue here, thanks!


WIP: Implement "b3 single" header format for zipkin

Signed-off-by: zyfjeff<EMAIL_ADDRESS>Description: Implement "b3 single" header format for zipkin
Risk Level: medium
Testing: unittest
Docs Changes: N/A
Release Notes: N/A
[Optional Fixes #Issue] #4289
[Optional Deprecated:]

@adriancole @objectiser PTAL if you have some time.
@zyfjeff can you add a release note for this and any relevant docs?

can do!
On Sun, 14 Oct 2018, 07:34 Matt Klein<EMAIL_ADDRESS>wrote:

@adriancole https://github.com/adriancole @objectiser
https://github.com/objectiser PTAL if you have some time.
@zyfjeff https://github.com/zyfjeff can you add a release note for this
and any relevant docs?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/envoyproxy/envoy/pull/4712#issuecomment-429583819,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAD61-9uo1V0gSPSyRMm2f9-7ZgT_zK8ks5ukniQgaJpZM4XaY5z
.

so ping me when some basic tests are in. It would be easiest to use the
example data in the b3 spec and show for example single header and multiple
headers with the same ID literals. thanks for the start
https://github.com/openzipkin/b3-propagation/blob/master/README.md#single-header
On Sun, 14 Oct 2018, 08:35 Adrian Cole<EMAIL_ADDRESS>wrote:

can do!
On Sun, 14 Oct 2018, 07:34 Matt Klein<EMAIL_ADDRESS>wrote:

@adriancole https://github.com/adriancole @objectiser
https://github.com/objectiser PTAL if you have some time.
@zyfjeff https://github.com/zyfjeff can you add a release note for
this and any relevant docs?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/envoyproxy/envoy/pull/4712#issuecomment-429583819,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAD61-9uo1V0gSPSyRMm2f9-7ZgT_zK8ks5ukniQgaJpZM4XaY5z
.

@adriancole  I have two questions want to consult with you?
Why FORMAT_MAX_LENGTH equal 32 + 1 + 16 + 2 + 16, not equal 32 + 1 + 16 + 2 + 1 + 16
https://github.com/openzipkin/brave/blob/master/brave/src/main/java/brave/propagation/B3SingleFormat.java#L45
If I received from downstream b3 header format, that spread to the upstream whether  change to b3 single header format ?

This is a bug in brave. thanks for noticing! I'll copy you on the fix.
On Wed, Oct 17, 2018 at 9:45 PM tianqian.zyf<EMAIL_ADDRESS>wrote:

@adriancole https://github.com/adriancole I have two questions want to
consult with you?
Why FORMAT_MAX_LENGTH equal 32 + 1 + 16 + 2 + 16, not equal 32 + 1 + 16 +
2 + 1 + 16
https://github.com/openzipkin/brave/blob/master/brave/src/main/java/brave/propagation/B3SingleFormat.java#L45
If I received from downstream b3 header format, that spread to the
upstream whether change to b3 single header format ?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/envoyproxy/envoy/pull/4712#issuecomment-430634088,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAD612AdcY4jZBk0OxuFMIcc7g_QEL7Mks5ulzRlgaJpZM4XaY5z
.

@adriancole If I received from downstream b3 header format, that spread to the upstream whether change to b3 single header format ?  this problem is not answer me yet

If I understand you, you are asking if you receive a "b3" header, should
you also send "b3" header?
I think for starters you only send "X-B3-*" headers because the most
important thing is to read "b3". You can make a setting to send both, but
it will cost a little more. I doubt sites will want to send only "b3" by
default.
So you can choose from:

send "X-B3-*" (no change downstream)
send both "X-B3-*" and "b3"
add option to control the behavior. Default to current, and allow people
to opt-in to sending both or just "b3".

maybe there is another choice
On Fri, Oct 19, 2018 at 4:47 PM tianqian.zyf<EMAIL_ADDRESS>wrote:

@adriancole https://github.com/adriancole If I received from downstream
b3 header format, that spread to the upstream whether change to b3 single
header format ? this problem is not answer me yet
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/envoyproxy/envoy/pull/4712#issuecomment-431291691,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAD615qPty4I-36Q0XgUWIPiiRCF_kkHks5umZGwgaJpZM4XaY5z
.

This PR default only send X-B3-*, the next PR add option to control the behaviors.

This PR default only send X-B3-*, the next PR add option to control the behaviors.

SGTM

@adriancole   @objectiser  I have already finished, can undertake codereview

@mattklein123 In which file do you add release note?

if you want to try this out in real life you can put your envoy around this
and propagate b3: tracid-spanid
https://github.com/openzipkin/brave-webmvc-example
check zipkin and see if it works end to end. I am good on this change.

@adriancole I found a bug
https://github.com/openzipkin/brave/blob/master/brave/src/main/java/brave/propagation/B3SingleFormat.java#L194
The following case will cause the if condition to be false, pos is not incremented and stays in the sampled position, causing the following to if (endIndex != pos + 17) return false
testcase:
  {
    Http::TestHeaderMapImpl request_headers{
        {"b3", fmt::format("{}-{}-d!{}", trace_id, span_id, parent_id)}};
    SpanContextExtractor extractor(request_headers);
    EXPECT_THROW_WITH_MESSAGE(extractor.extractSpanContext(true), ExtractorException,
                              "Invalid input: truncated");
  }

thanks again! I will look into it now
On Wed, 24 Oct 2018, 11:29 tianqian.zyf<EMAIL_ADDRESS>wrote:

@adriancole https://github.com/adriancole I found a bug
https://github.com/openzipkin/brave/blob/master/brave/src/main/java/brave/propagation/B3SingleFormat.java#L194
The following case will cause the if condition to be false, pos is not
incremented and stays in the sampled position, causing the following to if
(endIndex != pos + 17) return false
testcase:
{
Http::TestHeaderMapImpl request_headers{
{"b3", fmt::format("{}-{}-d!{}", trace_id, span_id, parent_id)}};
SpanContextExtractor extractor(request_headers);
EXPECT_THROW_WITH_MESSAGE(extractor.extractSpanContext(true), ExtractorException,
"Invalid input: truncated");
}
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/envoyproxy/envoy/pull/4712#issuecomment-432499819,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAD619bWkUJUIlrRxET3qrv5o6S08dSUks5un960gaJpZM4XaY5z
.

right we were making the log statement, though still falling back (as
opposed to raising an exception and breaking the request). To add log
verification in unit tests is annoying but worth it. will fix :P
On Wed, Oct 24, 2018 at 11:46 AM Adrian Cole<EMAIL_ADDRESS>wrote:

thanks again! I will look into it now
On Wed, 24 Oct 2018, 11:29 tianqian.zyf<EMAIL_ADDRESS>wrote:

@adriancole https://github.com/adriancole I found a bug
https://github.com/openzipkin/brave/blob/master/brave/src/main/java/brave/propagation/B3SingleFormat.java#L194
The following case will cause the if condition to be false, pos is not
incremented and stays in the sampled position, causing the following to if
(endIndex != pos + 17) return false
testcase:
{
Http::TestHeaderMapImpl request_headers{
{"b3", fmt::format("{}-{}-d!{}", trace_id, span_id, parent_id)}};
SpanContextExtractor extractor(request_headers);
EXPECT_THROW_WITH_MESSAGE(extractor.extractSpanContext(true), ExtractorException,
"Invalid input: truncated");
}
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/envoyproxy/envoy/pull/4712#issuecomment-432499819,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAD619bWkUJUIlrRxET3qrv5o6S08dSUks5un960gaJpZM4XaY5z
.

fixed in 5.4.4 you can also retry the example if you like. logs should be
prettier now
On Wed, Oct 24, 2018 at 11:29 AM tianqian.zyf<EMAIL_ADDRESS>wrote:

@adriancole https://github.com/adriancole I found a bug
https://github.com/openzipkin/brave/blob/master/brave/src/main/java/brave/propagation/B3SingleFormat.java#L194
The following case will cause the if condition to be false, pos is not
incremented and stays in the sampled position, causing the following to if
(endIndex != pos + 17) return false
testcase:
{
Http::TestHeaderMapImpl request_headers{
{"b3", fmt::format("{}-{}-d!{}", trace_id, span_id, parent_id)}};
SpanContextExtractor extractor(request_headers);
EXPECT_THROW_WITH_MESSAGE(extractor.extractSpanContext(true), ExtractorException,
"Invalid input: truncated");
}
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/envoyproxy/envoy/pull/4712#issuecomment-432499819,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAD619bWkUJUIlrRxET3qrv5o6S08dSUks5un960gaJpZM4XaY5z
.

@zyfjeff ps I've spoken to a number of sites who are not thrilled that envoy still emits data in zipkin v1 format. I think you can win even more friends if you have cycles to do v2 data format (json or proto3) as well. This would work in zipkin, but also clones of it like jaeger (who also accept zipkin v2 data) and all the myriad of proxies.

@zyfjeff ps I've spoken to a number of sites who are not thrilled that envoy still emits data in zipkin v1 format. I think you can win even more friends if you have cycles to do v2 data format (json or proto3) as well. This would work in zipkin, but also clones of it like jaeger (who also accept zipkin v2 data) and all the myriad of proxies.

I am not familiar with zipkin v2 data format, related documents?I can see first

Not sure if there are different docs but I did find https://zipkin.io/zipkin-api/#/ which appears to be v2.

I am not familiar with zipkin v2 data format, related documents?I can see
first

I opened a new issue. Hope you can help!
https://github.com/envoyproxy/envoy/issues/4839
I can help review and also point you at java code if you want.