Add dependency management, auto version based on git tags, and auto pypi publishing

Hello maintainers of Brax,
I really like your repo and would like to add tools to help with reproducing work in the future.
There are three parts of this PR.
Dependency management via poetry
poetry locks dependencies for better reproducibility. Another benefit is that I could pin the training related dependencies as dev dependencies, so that if the users (myself as an example) wants to use other DL framework to do training, they wouldn't have to also install tensorflow and other libraries unrelated to the env.
Auto version based on git tags via poetry-dynamic-versioning
So if you run poetry build, it's automatically going to create distributions like brax-0.0.0.post34.dev0+412962f.tar.gz. And if you draft a release / create a git tag like v0.0.1, it's going to create a distribution like brax-0.0.1. So this saves a lot of time from manually incrementing setup.py's versions.
Auto PyPi publishing,
If you add a PyPi API token to the repo's secret as follows, then every time you draft a release on Github, it's automatically going to push the corresponding release to PyPi.

These are the best practices I learned the hard way with my own env gym-microrts, which also started with something like pip install -e .  And at some point I had the most painful time trying to reproduce some past results, which resulted in the practices I am recommending to you now :)
For reference, I think the following snippet would work in Linux or Windows (not quite on Mac due to an upstream bug):
pip install gym_microrts==0.4.2 # make sure you have Java 8.0+ installed
curl -OL https://raw.githubusercontent.com/vwxyzjn/gym-microrts/4a8098811fc871a456041b3357f284359c466572/hello_world.py
python hello_world.py

I hope this might be useful to you and wish the best for your project.

@googlebot I signed it!

Hi Costa,
Thanks so much for opening this PR.  Having the pypi push connected to a git tag looks very handy, and I agree it would be nice to have optional dependencies like tensorflow for the learners.
To fit in with other google projects our ecosystem, rather than rely on poetry we will use bumpversion for the versioning support, and the extras_require that lives in setuptools for optional dependencies.
It's probably disappointing to hear this as your PR is fantastic!  poetry just doesn't fit some of our other workflows.  If you'd like to still make sure we get this packaging functionality in yourself, we'd be very happy to work with you on a new PR and I can show you a few pointers.
Otherwise, no sweat - we'll add this functionality in the coming weeks.  Thanks again!

Hi Erik,
Thanks for the prompt reply. Having tools that fit seamlessly with existing tool chains is definitely important! If possible, I would like to help contribute (which also helps my selfish dream to deprecate mujoco haha). If you could show me a few pointers, I could prepare another PR with bumpversion and others :)

Hi Costa - we ended up handling automating our release flow as it was something I had to learn myself, to be honest     :-).    so I would have done a poor job at requesting or understanding changes from others.
It's not too magical - you can see there's a github action now, like you had proposed, for pushing to pypi, and some of the versioning is now handled by internal tooling.  Thanks again for this PR - I appreciate that you jumped right in!

Hi @erikfrey, this is awesome! Thank you!


Ensure the log directory exists

Should address https://github.com/ManageIQ/manageiq/pull/17663#issuecomment-414452783

Closing in favor of https://github.com/ManageIQ/manageiq/pull/17886


[0.62.2] Using base64 strings for an Image URI crashes app

Description
I'm using base64 strings as image sources throughout my app. After upgrading react-native from 0.61.5 to 0.62.2, my app crashes immediately upon loading any image with the source as {"uri": 'base64encodedstring'}. There are no logs prior to the app crashing.  My app uses data:image/jpeg;base64 encoded strings.
React Native version:
System:
OS: macOS 10.15.4
CPU: (4) x64 Intel(R) Core(TM) i5-5250U CPU @ 1.60GHz
Memory: 101.43 MB / 8.00 GB
Shell: 5.7.1 - /bin/zsh
Binaries:
Node: 13.11.0 - ~/.nvm/versions/node/v13.11.0/bin/node
Yarn: 1.22.4 - /usr/local/bin/yarn
npm: 6.13.7 - ~/.nvm/versions/node/v13.11.0/bin/npm
Watchman: 4.9.0 - /usr/local/bin/watchman
Managers:
CocoaPods: 1.8.4 - /usr/local/bin/pod
SDKs:
iOS SDK:
Platforms: iOS 13.4, DriverKit 19.0, macOS 10.15, tvOS 13.4, watchOS 6.2
Android SDK:
API Levels: 27, 28, 29
Build Tools: 28.0.3, 29.0.2
System Images: android-26 | Google APIs Intel x86 Atom_64, android-28 | Google APIs Intel x86 Atom_64, android-29 | Google APIs Intel x86 Atom_64
Android NDK: Not Found
IDEs:
Android Studio: 3.6 AI-192.7<IP_ADDRESS>41897
Xcode: 11.4/11E146 - /usr/bin/xcodebuild
Languages:
Java: 1.8.0_232 - /usr/bin/javac
Python: 2.7.16 - /usr/bin/python
npmPackages:
@react-native-community/cli: Not Found
react: 16.11.0 => 16.11.0
react-native: 0.62.2 => 0.62.2
npmGlobalPackages:
react-native: Not Found
Steps To Reproduce
Provide a detailed list of steps that reproduce the issue.

Use {"uri": 'base64encodedstring'} as an Image source inside of a component

Expected Results
Image loads properly

Same issue here with rn v62.2

version 0.62.1 also affected

I've tested this issue on iOS

Same issue on ios @latest react-native

@daniel-sudz you are a hero.  TNX man.

I am using React Native 62.2 and changing versions['Flipper'] ||= '~> 0.37.0' solved my problem too. :)
JustCode
https://justnice.net/wp/blog/

Edit:
For anyone else that comes across this just change the version of Flipper in the podfile to
  versions['Flipper'] ||= '~> 0.37.0'

as this is what the pr did

You perhaps should change the version of Flipper-Folly
versions['Flipper-Folly'] ||= '~> 2.2'

update flipper Not working for me
  versions['Flipper'] ||= '~> 0.37.0'
  versions['DoubleConversion'] ||= '1.1.7'
  versions['Flipper-Folly'] ||= '~> 2.2'
  versions['Flipper-Glog'] ||= '0.3.6'
  versions['Flipper-PeerTalk'] ||= '~> 0.0.4'
  versions['Flipper-RSocket'] ||= '~> 1.1'

Along with the flipper update I also had to update,
  versions['Flipper-Folly'] ||= '~> 2.2'
  versions['Flipper-RSocket'] ||= '~> 1.1'

After updating versions in the pod file run
pod install --repo-update

Cheers!

I can also confirm the updating the version fixes this issue.

Thanks for the solution @daniel-sudz and @riteshpatel

@daniel-sudz Thanks a lot

Edit:
For anyone else that comes across this just change the version of Flipper in the podfile to
  versions['Flipper'] ||= '~> 0.37.0'

as this is what the pr did

the real contributor !

Edit:
For anyone else that comes across this just change the version of Flipper in the podfile to
  versions['Flipper'] ||= '~> 0.37.0'

as this is what the pr did

working fine with this changes

I get the following error when I try to upgrade flipper

Edit:
For anyone else that comes across this just change the version of Flipper in the podfile to
  versions['Flipper'] ||= '~> 0.37.0'

as this is what the pr did

I had the same problem here!
its worked for me <3

Along with the flipper update I also had to update,
  versions['Flipper-Folly'] ||= '~> 2.2'
  versions['Flipper-RSocket'] ||= '~> 1.1'

After updating versions in the pod file run
cd ios && pod install --repo-update && cd ..

Cheers!

Closing as we have a fix, thanks everyone

I get the following error when I try to upgrade flipper

Happen to me as well, when run pod install


feat(cart): trigger relay cart action when updateItem

Jira: https://coveord.atlassian.net/browse/LENS-1555
This branch is to emit the ec.cartAction when triggering the cartController.updateItem function. It's one of the event that will be implement within headless.
Demo of the result in Barca sport (local build linked to the website):
https://github.com/coveo/ui-kit/assets/58052881/55ebce50-e110-41e2-893f-bdc2d6af9b09

I think we already have a mechanism for triggering relay events from headless 🤔
Here's an example of creating an action that gets dispatched: https://github.com/coveo/ui-kit/blob/a38e686aaed83aa093055bb236bdd7e6c95b23fe/packages/headless/src/features/result/result-analytics-actions.ts#L22-L36
https://github.com/coveo/ui-kit/blob/8e7ec2490615b346aac692aa1c970a6d6b536a27/packages/headless/src/controllers/result-list/headless-interactive-result.ts#L49

I think we already have a mechanism for triggering relay events from headless 🤔 Here's an example of creating a relay analytics action that gets dispatched:
https://github.com/coveo/ui-kit/blob/a38e686aaed83aa093055bb236bdd7e6c95b23fe/packages/headless/src/features/result/result-analytics-actions.ts#L22-L36
https://github.com/coveo/ui-kit/blob/8e7ec2490615b346aac692aa1c970a6d6b536a27/packages/headless/src/controllers/result-list/headless-interactive-result.ts#L49

I'm taking a look into it, but I wonder if it complicity the implementation (it seems that the implementation that are using this mechanism has to handle legacy implementation as well. I don't it's the case with the cartController?)


Monitors are deprecated. Please use tf.train.SessionRunHook.

while I run the udc_train.py on tensorflow 1.1.0, it raise the error like that:
WARNING:tensorflow:From C:\Anaconda3\lib\site-packages\tensorflow\contrib\learn\python\learn\monitors.py:267: BaseMonitor.init (from tensorflow.contrib.learn.python.learn.monitors) is deprecated and will be removed after 2016-12-05.
Instructions for updating:
Monitors are deprecated. Please use tf.train.SessionRunHook.
Traceback (most recent call last):
File "E:/project/chatbot/udc_train.py", line 67, in
tf.app.run()
File "C:\Anaconda3\lib\site-packages\tensorflow\python\platform\app.py", line 48, in run
_sys.exit(main(_sys.argv[:1] + flags_passthrough))
File "E:/project/chatbot/udc_train.py", line 64, in main
estimator.fit(input_fn=input_fn_train, steps=None, monitors=[eval_monitor])
File "C:\Anaconda3\lib\site-packages\tensorflow\python\util\deprecation.py", line 281, in new_func
return func(*args, **kwargs)
File "C:\Anaconda3\lib\site-packages\tensorflow\contrib\learn\python\learn\estimators\estimator.py", line 430, in fit
loss = self._train_model(input_fn=input_fn, hooks=hooks)
File "C:\Anaconda3\lib\site-packages\tensorflow\contrib\learn\python\learn\estimators\estimator.py", line 927, in _train_model
model_fn_ops = self._get_train_ops(features, labels)
File "C:\Anaconda3\lib\site-packages\tensorflow\contrib\learn\python\learn\estimators\estimator.py", line 1132, in _get_train_ops
return self._call_model_fn(features, labels, model_fn_lib.ModeKeys.TRAIN)
File "C:\Anaconda3\lib\site-packages\tensorflow\contrib\learn\python\learn\estimators\estimator.py", line 1103, in _call_model_fn
model_fn_results = self._model_fn(features, labels, **kwargs)
File "E:\project\chatbot\udc_model.py", line 39, in model_fn
targets
File "E:\project\chatbot\models\dual_encoder.py", line 47, in dual_encoder_model
tf.concat(0, [context_embedded, utterance_embedded]),
File "C:\Anaconda3\lib\site-packages\tensorflow\python\ops\array_ops.py", line 1029, in concat
dtype=dtypes.int32).get_shape(
File "C:\Anaconda3\lib\site-packages\tensorflow\python\framework\ops.py", line 639, in convert_to_tensor
as_ref=False)
File "C:\Anaconda3\lib\site-packages\tensorflow\python\framework\ops.py", line 704, in internal_convert_to_tensor
ret = conversion_func(value, dtype=dtype, name=name, as_ref=as_ref)
File "C:\Anaconda3\lib\site-packages\tensorflow\python\framework\constant_op.py", line 113, in _constant_tensor_conversion_function
return constant(v, dtype=dtype, name=name)
File "C:\Anaconda3\lib\site-packages\tensorflow\python\framework\constant_op.py", line 102, in constant
tensor_util.make_tensor_proto(value, dtype=dtype, shape=shape, verify_shape=verify_shape))
File "C:\Anaconda3\lib\site-packages\tensorflow\python\framework\tensor_util.py", line 370, in make_tensor_proto
_AssertCompatible(values, dtype)
File "C:\Anaconda3\lib\site-packages\tensorflow\python\framework\tensor_util.py", line 302, in _AssertCompatible
(dtype.name, repr(mismatch), type(mismatch).name))
TypeError: Expected int32, got list containing Tensors of type '_Message' instead.
could you give me some advice, thank you

The order of arguments for tf.concat and tf.split have changed in TF > 0.12.
So, the actual error is in line 47 dual_encoder_model..
Replace these lines below ( # commented) and corrected below.
This should help..
Also, you may have to replace tf.batch_matmul  with tf.matmul  and
losses = tf.nn.sigmoid_cross_entropy_with_logits(logits=logits, labels=tf.to_float(targets))
rnn_outputs, rnn_states = tf.nn.dynamic_rnn(
cell,
#tf.concat(0, [context_embedded, utterance_embedded]),
tf.concat( [context_embedded, utterance_embedded], 0),
#sequence_length=tf.concat(0, [context_len, utterance_len]),
sequence_length=tf.concat([context_len, utterance_len], 0),
dtype=tf.float32)
#encoding_context, encoding_utterance = tf.split(0, 2, rnn_states.h)
encoding_context, encoding_utterance = tf.split(rnn_states.h, 2 , 0)

Fixing tf.concat does not fix this problem.

No, it fixes the problem in the trace above at line 47 in dual_encoder.py, which is due to a change in the order of arguments for tf.concat and tf.split in versions of TF > 0.12
There is a disconnect between the title of this issue and the actual stack trace.
The solution for Monitors being deprecated is to use tf.train.SessionRunHook. It's in the title itself.

No, it fixes the problem in the trace above at line 47 in dual_encoder.py,
which is due to a change in the order of arguments for tf.concat and
tf.split in versions of TF > 0.12
There is a disconnect between the title of this issue and the actual stack
trace.
The solution for Monitors being deprecated is to use
tf.train.SessionRunHook. It's in the title itself.
On Mon, Jul 17, 2017 at 2:20 PM, jayurbain<EMAIL_ADDRESS>wrote:

Fixing tf.concat does not fix this problem.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/dennybritz/chatbot-retrieval/issues/48#issuecomment-315888166,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AbzcOVjKpCgPKbU73VbD-CUKHJw5dnmnks5sO9ACgaJpZM4Nh-xr
.

How do you use tf.train.SessionRunHook ?
for instance I have :
hooks = [tf.train.LoggingTensorHook({'loss'}, every_n_iter = 4)]
learn.Experiment(estimator = estimator, train_input_fn = get_train_inputs, eval_input_fn = get_eval_inputs, train_monitors = hooks, eval_hooks = hooks)
It says: Monitors are deprecated. Please use tf.train.SessionRunHook., but I don't know how to change it.

I have same problem as @nikky78


1.24.0-beta3 Some grammars are not applied at start to opened editors

Prerequisites

[x] Put an X between the brackets on this line if you have done all of the following:

Reproduced the problem in Safe Mode: https://flight-manual.atom.io/hacking-atom/sections/debugging/#using-safe-mode
Followed all applicable steps in the debugging guide: https://flight-manual.atom.io/hacking-atom/sections/debugging/
Checked the FAQs on the message board for common solutions: https://discuss.atom.io/c/faq
Checked that your issue isn't already filed: https://github.com/issues?utf8=✓&q=is%3Aissue+user%3Aatom
Checked that there is not already an Atom package that provides the described functionality: https://atom.io/packages

Description
On a fresh Atom 1.24.0-beta3, grammars like the ones in language-babel do not get applied at start on already opened buffers.
Steps to Reproduce

Remove all atom configs.
Open atom and install language-babel
Open or create a .jsx file
Restart atom (ctrl + shift + f5 or close and open)

Expected behavior: The opened jsx file has the jsx grammar applied
Actual behavior: The opened jsx file has the NullGrammar (Plain Text)
Reproduces how often: 100%
Versions
Atom    : 1.24.0-beta3
Electron: 1.6.15
Chrome  : 56.0.2924.87
Node    : 7.4.0
OS: Archlinux
Additional Information
NB: When you close the file and open it again it works as expected.
It works with some other grammars.
Downstream issue: https://github.com/gandm/language-babel/issues/476

Thanks for the report - on macOS 10.12.6 I had 1.23.0-beta1 installed but couldn't reproduce but after updating to 1.24.0-beta3 I saw the behavior you described.

This seems to be resolved in atom 1.25.0-beta0

I think this was actually an issue with language-babel.


Create helplive.txt

The second domain for HELP University (helplive.edu.my)

@oktushka I am sorry for a delay with processing your request.
Pull request merged. Thank you!

@oktushka I am sorry for a delay with processing your request.
Pull request merged. Thank you!


Maven plugin

Hi, thx for the great linter!
It would be useful for some cases to use Zally as a maven plugin. I can submit a PR, if you don't mind.

Hi @baurchanu ,
Yes, sure. Good idea!

Hi, could someone review the PR, pls?

We have decided not to add Zally maven plugin directly to Zally. Check this PR comment for details.


RFC : Point In Time Search

Is your feature request related to a problem? Please describe.
Today, in OpenSearch, if you want to run different queries on the same data set chances are you will get different result as data is constantly changing. However, in real world scenario when analyzing data or trying to provide a consistent user experience to your end users you may want the result from a query not to change while the context remains the same and control when changes should appear in the result set. You want to be able to query the same data set and paginate through the data set expecting consistent result. This is not possible using current available options in OpenSearch.
Opensearch currently supports the following options to achieve pagination, each having a certain limitation:

Scroll API : Scroll API cannot share point in time context with other queries. Moreover, the scroll API only allows to move forwards(next page) in the search, cases when the client sends the request for a page but fails to get a response, a subsequent retry call skips the page(retried for) and returns the next page in the scroll.
Search After : The search_after mechanism doesn't preserve the state of data when the search was issued, so one can paginate using the key (search_after) and fetch subsequent pages while getting more recent results since the search was issued as the pagination progresses.
From To : This mechanism does not support deep pagination since every page request requires the shard to process all previous results and then filter the requested page which might be taxing deeper the pagination goes

Describe the solution you'd like
Point in Time allows users to run different queries against the same fixed data set in time. Point in time only takes data into account up until the moment it is created. Hence, none of the resources that are required to return the data from the initial request are modified or deleted. Segments are retained, even though the segment might already have been merged away and is not needed for the live data set. In short, Point in Time Search allows user to maintain a state which can be re-used by different queries in order to achieve consistent results.
Key goals:

Optimize resource consumption compared to a scroll by providing a consistent, shareable view of data set across queries. More segments are otherwise needed to be retained as needed by individual queries which means more file handles, more disk and more heap to keep metadata from segments in the heap.
Resilient to

Network failures : allows searches to move forward with a search_after parameter
Shard failures for read-only data : allows retries on other shard copies that share the same segments (Phase - II)

Replaces scroll API, as a more comprehensive solution for deep pagination when used with search_after
Point in Time will be supported by Asynchronous Search and Cross Cluster searches

APIs
Create Point In Time API
Unlike a Scroll, by creating a dedicated Point in Time, we decouple the context from a single query and make it re-usable across arbitrary search requests by passing the Point in Time Id. We can achieve this by using the Create Point in Time API.
POST <index>/_point_in_time?keep_alive=1m
{
   "id" : "s9O9QAIFaW5kZXgWOFVaMXFTc3pTV3lLMGE4VU42dmo4dwAWekthUVBmYnRUWk9XVzh4WW56TG5lZwAAAAAAAAAAARZQd3JkNlE4WlJicXRuS0M1VzNDaHV3BWluZGV4FjhVWjFxU3N6U1d5SzBhOFVONnZqOHcBFnpLYVFQZmJ0VFpPV1c4eFluekxuZWcAAAAAAAAAAAIWUHdyZDZROFpSYnF0bktDNVczQ2h1dwEWOFVaMXFTc3pTV3lLMGE4VU42dmo4dwAA",
   "created_time" :<PHONE_NUMBER>283,
   "end_time" :<PHONE_NUMBER>283
}

Delete Point In Time API
Point-in-times are automatically closed when the keep_alive is elapsed. However, keeping point-in-times has a cost; hence, point-in-times should be closed as soon as they are no longer used in search requests. We may also delete a Point in Time and free the resources before its keep alive using the Delete Point in Time API.
DELETE /_point_in_time/<id>
List All Active Point In Time API
A useful admin API to have is to list all active Points in Time and their keep-alives.
GET /_point_in_time

[
    {
        "point_in_time_id_1",
        "created_time" :<PHONE_NUMBER>283,
        "end_time" :<PHONE_NUMBER>283
    },
    {
        "point_in_time_id_2",
        "created_time" :<PHONE_NUMBER>272,
        "end_time" :<PHONE_NUMBER>283
    }
    ...
    ...
]

Using a Point in Time in a search request:
In the search request we pass the point in time id and (optionally) a keep alive to extend the Point In Time. (Passing PIT id in search request is supported in Opensearch)
Search request with PIT ID will not accept indices, preference, routing and indices options as these are already passed at the time of creating a Point In Time.
GET /_search
{
  "pit": {
        "id":  "ID_RETURNED_FROM_CREATE_POINT_IN_TIME_REQUEST",
        "keep_alive": "1m" //optional to extend a Point In Time
  },
  "sort": [
    {
      "name.keyword": {
        "order": "desc"
      }
    }
  ],
  "search_after" : ["Opensearch", 1] //optional to fetch further results
}

With constantly changing data set, queries fired at different times return different results. So if we needed to get consistent results for queries that were fired at different times(even back to back) for a changing data set, we don’t have a mechanism in place. In short Point in Time allows user to maintain a state which can be re-used by different queries in order to achieve consistent results
Then we have the need to paginate across a data set. The best mechanism to paginate so far is a scroll API, other than search_after that allows for consistent results across different scrolls for the same search query but works on live data.
Problems with pagination

Scroll API : The scroll API only allows to move forward(next page) in the search, cases when the client sends the request for a page but fails to get a response, a subsequent retry call skips the page(retried for) and returns the next page in the scroll. Scroll API cannot share point in time context with other queries.
Search After : The search_after mechanism doesn't preserve the state of data when the search was issued, so one can paginate using the key (search_after) and fetch subsequent pages while getting more recent results since the search was issued as the pagination progresses.
From To : This mechanism does not support deep pagination since every page request requires the shard to process all previous results and then filter the requested page which might be taxing deeper the pagination goes

Hi @rajkthakur, are you actively working on this issue? if yes, please could you assign it to yourself and add a comment?

@anasalkouz I am actively working on this issue.

Can we mark this as a proposal?

I have done a small POC to check feasibility of the proposed APIs
We will not be able to provide a List All Points In Time APIs, as a point in time is not tied to any coordinator node.
Rather the Point In Time ID is simply a Base 64 encoded hash of list of reader context to node mappings and a UUID.

We will provide new information in the Nodes stats api -
a nested object in the search section mentioning the number of currently active Point In Time contexts, total number of Point In Time contexts.
This will help keep track of point in time related statistics.

We will provide settings to:

restrict the max open point in time contexts on a node
restrict the maximum keep alive allowed for point in times.

Currently doing a POC to add an API for PIT disk utilization i.e. segments retained by PIT Ids. I am trying to put out stats similar to cat segments, but for Points In Time.

Have verified that its possible to get PIT segments info on the lines of cat segments api.
https://github.com/eirsep/OpenSearch/tree/pit_poc
It would give important insights into a PIT's disk utilization.

Segments are retained

This can get expensive. I see the objective is to optimize resource consumption :heavy_check_mark: .  I think this is a specialized use case for archival or time based analysis use cases and not the normal search use case so this should be configured through an Index Scoped setting.
Segment replication will heavily change this design as it will push a lot of the PIT burden on the storage layer.

Segments are retained

This can get expensive. I see the objective is to optimize resource consumption ✔ . I think this is a specialized use case for archival or time based analysis use cases and not the normal search use case so this should be configured through an Index Scoped setting.

I agree that segment retention is expensive, but this is exactly what scrolls do today.  The resource consumption is being optimised only in comparison to Scrolls(i.e. being able to share PIT Id across queries while scrolls create different contexts per scroll).
(Among other things) PIT would be a replacement for scrolls and hence be the pagination solution for Opensearch, which is also an important use case to keep in mind.
Will look into how-to and benefits of configuring PIT through Index Scoped setting.

Segment replication will heavily change this design as it will push a lot of the PIT burden on the storage layer.

Segments retained by PIT/Scrolls will not be replicated I think. Can you plz elaborate the caveats you see?

PIT would be a replacement for scrolls

:+1:

Can you plz elaborate the caveats you see?

A storage engine w/ verisoned backups (e.g., S3 buckets w/ versioning) can be used to restore files from backups. I think this along w/ Lucene sequence IDs enables this feature without having to retain as many historic segments. A storage engine w/o versioning (e.g., local NFS, or smb) could possibly use the segment retention logic provided by this feature.

Can you plz elaborate the caveats you see?

A storage engine w/ verisoned backups (e.g., S3 buckets w/ versioning) can be used to restore files from backups. I think this along w/ Lucene sequence IDs enables this feature without having to retain as many historic segments. A storage engine w/o versioning (e.g., local NFS, or smb) could possibly use the segment retention logic provided by this feature.

Ack.
Thanks for elaborating! So, segment replication feature would have to handle how Scrolls would function which is currently doing segment retention (prolly by using versioned backups as you've mentioned). Hence, by extension, PITs will get handled too as PITs are simply re-using that idea.

@CEHENKLE Can you please create a feature branch feature/point-in-time which will be used to run full test suite?

https://github.com/opensearch-project/OpenSearch/tree/feature/point-in-time created

@rramachand21 is this still aiming for 2.0?

I think this is a specialized use case for archival or time based analysis use cases and not the normal search use case

I'm reading this feature as primarily a replacement/improvement over the scroll API. It seems like the primary use case is for pagination (which is currently solved by the scroll API but does have limitations). This essentially generalizes it a bit and gives semantics similar to snapshot isolation in a traditional database where you can do multiple queries within a transaction and observe a consistent view of the data. @nknize do you have any major concerns moving forward with this feature?
I do have a nitpick about the name, though, particularly the /_point_in_time API. "Point in time" is likely a phrase to be overloaded in the future, first and most obvious to me is something like "point in time restore". There also may in fact be other features to solve archival or time-based analysis requirements that would require the ability to do searches from points far in the past. I'd love to hear opinions from other folks, but given that this feature's scope is pretty narrowly focused on "establish a point in time now and allow me to do some searches against it for a very limited duration" it might be a good idea to give it a name that won't conflict/confuse with future point-in-time-related features.

Scroll API cannot share point in time context with other queries

The inability to share point-in-time contexts is mentioned several times, but what is the use case for sharing these contexts? The pagination use case makes total sense but I don't think that generally requires sharing the context. The ability to restrict maximum keep-alive duration also makes a lot of sense to put a cap on worst-case resource consumption. However, that limit is likely to be in tension with the usefulness of sharing the contexts if they are short-lived, so I'm curious about the use cases that are motivating the share-ability requirement.

I think the major use case of share-ability is the ability to execute different types of queries and derive better insights on the same consistent view of data. Then some queries might fail or timeout due to various reasons. Once PIT is created retries become simpler as it allows user to resume queries on the same view

+1 for /_search/_point_in_time

I agree with @anasalkouz and @andrross. /_search/_point_in_time is more clear.

@elfisher @anasalkouz @andrross I like and agree with the naming of /_search/_point_in_time. But there is one decision we need to make here based on the existing field "pit" (coming from legacy code in 7.x version) in the search request that customers managing their own 7.x clusters would be using and when they migrate to OpenSearch, they will need to make changes to move to this new API /_search/_point_in_time that we plan to provide.
Initial thought of keeping it "pit" was coming from the above data. If we are ok with this breaking change, we all can agree on this and go ahead with /_search/_point_in_time
For reference on existing field from 7.x code, please take a look at https://github.com/opensearch-project/OpenSearch/blob/main/server/src/main/java/org/opensearch/search/builder/SearchSourceBuilder.java#L128

@rramachand21 Could you elaborate more? what are the breaking changes? my understanding, we are introducing new feature and API that never been used before.

Here is the UX proposal for Point in time search:  [UX] Point in time in OpenSearch Dashboards (WIP) #3415, please review and comments, thanks.🤗

@Bukhtawar How the PIT query handle deleted documents in the segment.
For example, if a PIT query was created at time t,  search would skip all deleted documents in the segments irrespective of whether it was marked as deleted before or after the PIT creation time t right? Just wanted to know how we're handling it.

Can we add labels for "roadmap" and the version of OpenSearch this is targeting? I can add it to the overall project roadmap in the right column once that is done.

@rajkthakur I'm getting this error { Message: "Your request: '/_pit' is not allowed." } when querying AWS OpenSearch, that it should support releases 1.3, 1.2, 1.1, 1.0. Which OS version has this merge?

@rajkthakur I'm getting this error { Message: "Your request: '/_pit' is not allowed." } when querying AWS OpenSearch, that it should support releases 1.3, 1.2, 1.1, 1.0. According to latest AWS annoucement they have deployed OS 1.3, but there are any details about PiT support. Which OS version has this PR?

@loretoparisi AWS OpenSearch will support point in time search in upcoming release. It is not supported in AWS OpenSearch 1.3. Point-in-time search will be supported in OpenSource OpenSearch 2.3.0 release.

@rajkthakur: I see this issue is labeled for v2.3.0 release, which has code freeze today i.e. Sep 7. I see open backport PRs to 2.x. Can you please prioritize review/merge.

Hi @rajkthakur, just checking in from the security team to see if there is anything you need finalized from us before the upcoming 2.4 freeze.

@rajkthakur do you still track this for 2.4 release? code freeze on 11/3

@anasalkouz Yes we are tracking this for 2.4 release. Documentation is pending, we will be closing it before 11/3.
@bharath-techie Please check with @rajkthakur and close this by next week

Documentation PR - https://github.com/opensearch-project/documentation-website/pull/1753

supported

@dhruv16dhr I've recently attempted again to use PIT using AWS OpenSearch / Kibana 2.5, but I'm getting
{ Message: "Your request: '/_pit' is not allowed." }

the currently installed version of AWS OS is
"version" : {
    "number" : "7.10.2",
    "build_snapshot" : false,
    "lucene_version" : "9.4.2",
    "minimum_wire_compatibility_version" : "7.10.0",
    "minimum_index_compatibility_version" : "7.0.0"
  }


How to handle alias-less channels?

At first blush, I guess it seems pragmatic to just wait for initial sync to finish, then talk to the IRC client using the raw channel IDs.
If a channel picks up an alias at some point in the future, how do we make renaming a channel like that not suck?

i'd just create a new channel whenever the human-readable name changes - it happens quite infrequently after all. one can always playback some scrollback in the new channel for continuity, and warn that the old one has disappeared from under you in the scrollback there.
@richvdh: i'm totally failing to find the spec you wrote on human readable naming rules for channels (when you were working on pidgin) - any idea where it went?

@richvdh: i'm totally failing to find the spec you wrote on human readable naming rules for channels (when you were working on pidgin) - any idea where it went?

https://matrix.org/docs/spec/r0.0.1/client_server.html#calculating-the-display-name-for-a-room


Fixed typo in file name.

sortedcombied-knock-dnsrecon-fierce-reconng.txt -> sortedcombined-knock-dnsrecon-fierce-reconng.txt

Thank you!


Seeing set-env error while running my workflow with the latest version of functions-action.

I am seeing this error message while running my workflow using the latest version of this action:
Error: The set-env command is deprecated and will be disabled on November 16th. Please upgrade to using Environment Files.
When are the new version going to be released to fix it? It seems that the action will be deprecated next week,

@MohdmM01 Are you facing this issue while using azure/functions-action@v1? Can you please share the workflow used?

I have a couple of workflows that use azure/functions-action@v1 that have stopped working in the last day or so - and I was just in the process of raising a question on stack overflow, when I saw this.  i do recall mention of set-env in the logs a few days ago but I was focusing on something else so didn't follow-up.
I have a number of applications that use this action, and as it happens they have all stopped working overnight.  I suspect they may be related to this.  There have been no other relevant changes in my repos.  Take a look at these:
https://github.com/nikkh/heracles-ai/runs/1411302955
https://github.com/nikkh/heracles-ai/runs/1406173184
https://github.com/nikkh/zodiac/runs/1407855810
This is a log from a successful run for the first failure abive:
https://github.com/nikkh/heracles-ai/runs/1397235421
Unfortunately, since I am mainly developing the workflows I have a cleanup job that deletes successfully deployed azure resources so I cant go back and look at (e.g.) kudo logs for successful deployments.
I'm an MS-FTE you can ping nhill to get in touch.

Had a call with @nikkh and tried to reproduce the issue. we did not face the error set-env command deprecated.

@MohdmM01 please confirm if you are still facing the issue

@MohdmM01 Can you please verify if you are still facing the issue?

I am no longer seeing the issue as this action is relying on other actions that has been updated to not use set-env. Thanks for the support.


Where can I find NF4 info?

Is there more info on NF4 available?
How does NF4 quantisation work. Is there any reweighting of matrices after quantising, eg using a Hessian of the loss?

answered here


iOS 14: calendar view for date selection

Is your feature request related to a problem? Please describe.
Nope
Describe the solution you'd like
When using mode: 'date', I would like to be able to have a calendar view instead of the wheel, and a fallback to wheel on < iOS 14
Describe alternatives you've considered
other libraries, like duet-datepicker
Additional context
https://www.swiftpal.io/articles/new-uidatepicker-in-ios-14

Basicly this like should be editable by some input config:
self.picker?.preferredDatePickerStyle = UIDatePickerStyle.wheels

I am working on this for a new version of the datepicker, this is available in v0.3 which will be used in conjunction with the capacitor v3

Cool! Got any idea on release date?

I'm currently a little busy, so I think that someday in May I should release this realease.
But if you install it with npm i<EMAIL_ADDRESS>you can test it.

I guess the 0.3.0 version only works with capacitor 3+ as well? since the datepicker is telling me
"export 'registerPlugin' was not found in '@capacitor/core' :)

Hi, just to come back. After following the capacitor update to 3.0 guide and inserting the community datepicker beta version, it seems to work on my iphone.

One little bug that may or not may be discovered yet: i pass locale NL: this causes the month to show dutch indeed, but the days remain in english whatever locale i pass.

@danielprrazevedo, Updated my capacitor and inserted beta version of datepicker. it works.
One little bug: passing a locale does update the month value, but not the day values.

Can I know which version of @capacitor-community/date-picker that you are using? Because I am facing issue in iOS device by using v0.2.3


fix(dataTable) : filter by label now working(WIP)

Now Clicking on a Lable in the Tabel View filters the WorkItem List by Label.
Please Refer: https://github.com/openshiftio/openshift.io/issues/2447
Test: WIP

Can one of the admins verify this patch?

[test]

Codecov Report

Merging #2516 into master will decrease coverage by 0.01%.
The diff coverage is 0%.

@@            Coverage Diff            @@
##           master   #2516      +/-   ##
=========================================
- Coverage    9.95%   9.93%   -0.02%
=========================================
  Files         193     193
  Lines       12522   12542      +20
  Branches     1761    1764       +3
=========================================
  Hits         1246    1246
- Misses      11207   11227      +20
  Partials       69      69

Impacted Files
Coverage Δ

src/app/components_ngrx/labels/labels.component.ts
3.57% <0%> (-1.99%)
:arrow_down:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 4c7e829...965d81d. Read the comment docs.

@divyanshiGupta fabric8/fabric8-ui:SNAPSHOT-PR-2516-2 is deployed and available for testing at https://fabric8-ui-fabric8-ui-pr-2516-fabric8-planner.badger.fabric8.io

@divyanshiGupta fabric8/fabric8-ui:SNAPSHOT-PR-2516-3 is deployed and available for testing at https://fabric8-ui-fabric8-ui-pr-2516-fabric8-planner.badger.fabric8.io

Somehow the code that I pushed is not available in the generated snapshots. I have pushed a new commit to rebuild the snapshot.

@divyanshiGupta fabric8/fabric8-ui:SNAPSHOT-PR-2516-5 is deployed and available for testing at https://fabric8-ui-fabric8-ui-pr-2516-fabric8-planner.badger.fabric8.io

@divyanshiGupta Your image is available in the registry. Run docker pull registry.devshift.net/fabric8-ui/fabric8-planner:SNAPSHOT-PR-2516 && docker run -it -p 5000:8080 registry.devshift.net/fabric8-ui/fabric8-planner:SNAPSHOT-PR-2516 and visit http://localhost:5000 to access it.

Ike Plugins (test-keeper)
Thank you @divyanshiGupta for this contribution!

It appears that no tests have been added or updated in this PR.
Automated tests give us confidence in shipping reliable software. Please add some as part of this change.
If you are an admin and you are sure that no test is needed then you can use the command /ok-without-tests as a comment to make the status green.
For more information please head over to official documentation. You can find there how to configure the plugin - for example exclude certain file types so if PR contains only them it won't be checked.

@divyanshiGupta Your image is available in the registry. Run docker pull registry.devshift.net/fabric8-ui/fabric8-planner:SNAPSHOT-PR-2516 && docker run -it -p 5000:8080 registry.devshift.net/fabric8-ui/fabric8-planner:SNAPSHOT-PR-2516 and visit http://localhost:5000 to access it.

@divyanshiGupta Your image is available in the registry. Run docker pull registry.devshift.net/fabric8-ui/fabric8-planner:SNAPSHOT-PR-2516 && docker run -it -p 5000:8080 registry.devshift.net/fabric8-ui/fabric8-planner:SNAPSHOT-PR-2516 and visit http://localhost:5000 to access it.

@divyanshiGupta Your image is available in the registry. Run docker pull registry.devshift.net/fabric8-ui/fabric8-planner:SNAPSHOT-PR-2516 && docker run -it -p 5000:8080 registry.devshift.net/fabric8-ui/fabric8-planner:SNAPSHOT-PR-2516 and visit http://localhost:5000 to access it.

@divyanshiGupta Your image is available in the registry. Run docker pull registry.devshift.net/fabric8-ui/fabric8-planner:SNAPSHOT-PR-2516 && docker run -it -p 5000:8080 registry.devshift.net/fabric8-ui/fabric8-planner:SNAPSHOT-PR-2516 and visit http://localhost:5000 to access it.

@Raunak1203 I have made the changes according to the review

@divyanshiGupta Your image is available in the registry. Run docker pull registry.devshift.net/fabric8-ui/fabric8-planner:SNAPSHOT-PR-2516 && docker run -it -p 5000:8080 registry.devshift.net/fabric8-ui/fabric8-planner:SNAPSHOT-PR-2516 and visit http://localhost:5000 to access it.

@divyanshiGupta Your image is available in the registry. Run docker pull registry.devshift.net/fabric8-ui/fabric8-planner:SNAPSHOT-PR-2516 && docker run -it -p 5000:8080 registry.devshift.net/fabric8-ui/fabric8-planner:SNAPSHOT-PR-2516 and visit http://localhost:5000 to access it.

@Raunak1203 it is supposed to be like that. The list is supposed to get filtered by label when we click a label from quick preview as well

@divyanshiGupta i am clicking on cross to remove label

@Raunak1203 fixed

@divyanshiGupta Your image is available in the registry. Run docker pull registry.devshift.net/fabric8-ui/fabric8-planner:SNAPSHOT-PR-2516 && docker run -it -p 5000:8080 registry.devshift.net/fabric8-ui/fabric8-planner:SNAPSHOT-PR-2516 and visit http://localhost:5000 to access it.

@divyanshiGupta Your image is available in the registry. Run docker pull registry.devshift.net/fabric8-ui/fabric8-planner:SNAPSHOT-PR-2516 && docker run -it -p 5000:8080 registry.devshift.net/fabric8-ui/fabric8-planner:SNAPSHOT-PR-2516 and visit http://localhost:5000 to access it.

@divyanshiGupta please check the assigned label in the detail page , it shows 2 cross

@divyanshiGupta fabric8/fabric8-ui:SNAPSHOT-PR-2516-17 is deployed and available for testing at https://fabric8-ui-fabric8-ui-pr-2516-fabric8-planner.badger.fabric8.io

@divyanshiGupta fabric8/fabric8-ui:SNAPSHOT-PR-2516-18 is deployed and available for testing at https://fabric8-ui-fabric8-ui-pr-2516-fabric8-planner.badger.fabric8.io

@divyanshiGupta Your image is available in the registry. Run docker pull registry.devshift.net/fabric8-ui/fabric8-planner:SNAPSHOT-PR-2516 && docker run -it -p 5000:8080 registry.devshift.net/fabric8-ui/fabric8-planner:SNAPSHOT-PR-2516 and visit http://localhost:5000 to access it.

@divyanshiGupta fabric8/fabric8-ui:SNAPSHOT-PR-2516-19 is deployed and available for testing at https://fabric8-ui-fabric8-ui-pr-2516-fabric8-planner.badger.fabric8.io

@divyanshiGupta Your image is available in the registry. Run docker pull registry.devshift.net/fabric8-ui/fabric8-planner:SNAPSHOT-PR-2516 && docker run -it -p 5000:8080 registry.devshift.net/fabric8-ui/fabric8-planner:SNAPSHOT-PR-2516 and visit http://localhost:5000 to access it.

@divyanshiGupta Your image is available in the registry. Run docker pull registry.devshift.net/fabric8-ui/fabric8-planner:SNAPSHOT-PR-2516 && docker run -it -p 5000:8080 registry.devshift.net/fabric8-ui/fabric8-planner:SNAPSHOT-PR-2516 and visit http://localhost:5000 to access it.


RFE: pre/post blocks for subcommands

I'm working on a an app with a lot of subcommands.
It would be great if a pre block could be specified inside a command to define a pre-block that is executed for all subcommands of that command.
desc "manage remotes"
command :remote do |c|
  c.command :repo do |repo|
    # do something
  end
  c.command :other_subcommand do |other|
    # do something else
  end
  pre do |global,command,options,args|
    # This gets done for both app remote repo and and app remote other_subcommand
    true
  end
end

You could mimic this by using the passed command object in the pre.  It has a parent attribute which is either the parent command or the GLI App if the command is a top level subcommand, so you could do something like:
def find_top_level_command(command)
  if command.parent.kind_of?(GLI::Command)
    find_top_level_command(command.parent)
  else
    command
  end
end

pre do |global,command,options,args|
  top_level_command = find_top_level_command(command)
  if top_level_command.name == "foo"
    # do whatever common stuff is needed for "foo" commands
  end
end

Hack-ish, but would work.

Thank you! This gets the job done!


redirect_uri is hardcoded for https

This affects github.js however it does/may affect other Oauth endpoints. Currently the code adds &redirect_uri=https%3A%2F%2F[Rest of url] to the url that the user is directed to at github. When this redirect_url isn't the same as the one specified in github's settings it will cause an error, and the github will redirect you back to your app with the error:
horizon_error=The%20redirect_uri%20MUST%20match%20the%20registered%20callback%20URL%20for%20this%20application.
This causes an issue in development when working with local addresses where you need to use http (for example with a local address). I suggest that we change line 108 in utils.js to use the page's current protocol, although I'm not sure how you would want to go about doing that.
My other suggestion is to change line 28 in github.js to not include the redirect_url option, as it is optional to use with github, and it is configurable on github's website. Again this may or may not apply to other oath providers, I'm not super familiar with Oauth.
Now I made this change locally, however after logging in with github I get sent to http://localhost:8181/?horizon_error=session%20expired, I'm not sure if this is related.

I may be wrong, but the documentation for most (if not all) auth providers say that you should only use HTTPS for the redirect URI.  At least some of them will change http to https or reject your redirect URI if it is not secure.  Out of security concerns, I'm not convinced we should allow http redirect URIs in the release branch.  Omitting the redirect URI if it is not specified seems fine, as long as the provider handles that correctly.
Keep in mind that it is not too difficult to use HTTPS even when developing on localhost - using a self-signed key, you just need to tell your browser to ignore the warning.

If you want to force https during production, great. But forcing it during development is an unnecessary burden. No other library I've encountered has forced me to use https for localhost.
Although you say it's "not too difficult", it is still an unnecessary, de-motivational blocker, especially when you're just trying to see if horizon works at all on your own computer.

I agree that we should allow for http redirect URLs. We can print a warning maybe if we detect it, but I don't see a good reason for forbidding it completely.
As @mindeavor says, forcing https just adds another step to getting a test application up. And it's not a particularly intuitive step either (you'll have to look up documentation for how to enable encryption at the very least, plus figure out hot wo create self-signed keys, how to convince hz in dev-mode to enable encryption (https://github.com/rethinkdb/horizon/issues/643) etc.).

Moving this to first post-2.0 release.

This is actually kind of a roadblock for development thru proxy.
I created a Vue.js app, and I'm using Webpack with proxy configuration, where the main Horizon backend act as a normal backend and auth. I'm developing for a Facebook auth right now, and it's forced with a https://<IP_ADDRESS>:8181/... redirect_uri, whereas my main Webpack app is at http://localhost:8080. The Webpack is important because it provides nice features right-out-of-the-box, and the most important for me is the live reload with hot patching, I mean productivity and other nice features makes developer happy.
The current situation makes it hard to develop in this kind of scenario.
Obviously, I'm not the only one who is doing this kind of development, I'm sure there are others. But right now, few here have reported this.
Since Horizon is on the rise, roadblocks like this may turn-away people who should have actually adopted Horizon. I believe Horizon is great, but hardcoded things like these, and forcing HTTPS during development is just plain demotivating.
Convention is nice, but there should be a good reason should it be enforced. It's development on localhost and not production.

Because I needed to go further anyhow, I can now confirm, that even after HTTPS is enabled, it would still redirect to https://<IP_ADDRESS>:8181/?horizon_error=session%20expired#_=_ with an error.
Not sure if that's a confirmed bug.

Hey, friendly ping any news? :)

@Huvik If you are looking for a quick fix right now try changing line 108 in utils.js to what you need


Need Query Promotions functionality to use Flask-RESTX

As a developer
I need the query Promotions functionality of my microservice to use Flask-RESTX
So that the code for this functionality is easier to maintain and debug using Swagger
Details and assumptions

Part of coverage for #49; assumptions from that story apply here as well

Acceptance criteria
Given [some initial state]
When [some event]
Then [some measurable outcome]

part of promotion collection class for flask-restx; code is co-resident with list all promotions code inside the class


First set of updates for CELA compliancy

Removing the manufacturer field as we don't want manufacturers displayed.
Updated the data.json file to be CELA compliant.
Updated the weight/height in kg/m to have more realistic values.

Part of the new data is being reviewed by CELA, so this PR won't be merged until next week. Also, I don't want to push anything while John is demoing the app in VS Live conference. :)


Check for API breaking changes in CI

As we move towards a 1.0 release it would be nice to check CI that we don't make breaking changes. Consider using

https://github.com/Enselic/cargo-public-api
https://github.com/obi1kenobi/cargo-semver-checks

We should definitely be using cargo-semver-checks -- though bearing in mind that it's still very incomplete and we can't blindly trust an "ok" from it.
cargo-public-api looks pretty interesting. I like the idea of committing the public API in contrib/ or somethig and then diffing against that in CI, so that any API changes become visible (and easy to check for after-the-fact). And maybe there are lints we could run against the text file to check if e.g. all errors impl Debug and std::Error and so on. We'd have to experiment a bit to see how well it handles our feature-gating etc.

Whats the status of this one crew, do we want it in before next release? The one on bech32 has stalled.
ref: https://github.com/rust-bitcoin/rust-bech32/pull/141

I'm not sure there should be rush but I want it eventually, definitely before 1.0 of any crate. Doing it sooner has the advantage of being able to auto-label PRs.

Let's just rebase #1880 and do it here, since Kix is onboard.
The history here is that we tried opening #1880 and got no reply from Kix after several weeks and were unable to move forward. So we switched to doing it in leaf crates where we didn't need Kix's approval (e.g. in rust-secp https://github.com/rust-bitcoin/rust-secp256k1/pull/656 and in rust-bech32). Then in rust-bech32 we have gotten no reply from Clark for several weeks :).
For what it's worth, it's worked great in rust-secp ... but that's not worth much since the API for rust-secp never changes :).
But since Kix is online and in favor of this, I think we should just pull the trigger instead of waiting on rust-bech32.

Then in rust-bech32 we have gotten no reply from Clark for several weeks

Just to clear the respond Clark did respond, it was back waiting on @Kixunil (because of merge strategy proposed by me). No one is really at fault here.
Anyways, I'll rebase and YOLO - if there are too many teething problems we can always revert.

I don't think this is really closed, the title suggests checking breaking changes, #2713 just "displays" all changes and we have no tool to highlight the breaking ones..

Yeah, we should add cargo-semver-checks, but we should try to avoid a false sense of security because it's really incomplete.
Though thankfully, the stuff it misses is often obvious to a human, and the stuff it catches is often really non-obvious.
The other issue is: what happens if there are breaking changes? Maybe CI should require an [API-BREAK] prefix or something in every PR that breaks the API? Because we don't want to forbid changes entirely (yet).

because it's really incomplete.

There are so many weird semver footguns that it's probably already better than humans. And even if it isn't, just catching a few non-obvious ones makes it useful anyway.

Maybe CI should require an [API-BREAK] prefix or something

Yeah, I like that. We have the API break label but we need to set it before CI runs. That's probably doable for maintainers but not others. Maybe we could allow saying [API-BREAK] in commit message which sets the label before the check runs.
Also the label should invert the check so that if someone claims to do API-break but it doesn't we detect it (though maybe it'll end up as a bug report at cargo-semver-checks).

Hi all! I'm the cargo-semver-checks maintainer 👋 Please ping me if you run into any issues.

There are so many weird semver footguns that it's probably already better than humans.

It's already way better than humans. We have the data to prove it: https://predr.ag/blog/semver-violations-are-common-better-tooling-is-the-answer/
That study found that more than 1 in 6 of the top 1000 most-downloaded Rust crates has violated semver in a way cargo-semver-checks could have caught and prevented. We went to extreme lengths to validate those numbers: as just one example, for each semver violation we claimed, we produced a "witness" program that relied on the broken functionality and then validated that it indeed suffers a compile error as a result of upgrading to the broken release.
That study is a year and a half old. cargo-semver-checks today has almost twice as many lints as it did at that time, so it's gotten much better at catching breakage since then. If we re-ran that study, we probably would find even more semver violations that we missed the first time around.

Maybe we could allow saying [API-BREAK] in commit message which sets the label before the check runs. Also the label should invert the check so that if someone claims to do API-break but it doesn't we detect it (though maybe it'll end up as a bug report at cargo-semver-checks).

Small nit: we consider undetected (false-negative) semver violations as a missing feature (we have a list), not a bug. We do not and cannot promise to catch everything — the pesky Halting Problem is in the way.
There are several categories of issues where cargo-semver-checks is known to miss things: https://predr.ag/blog/four-challenges-cargo-semver-checks-has-yet-to-tackle/
None of those are fundamental problems, and all of them are solvable with more funding and judicious application of engineering. If you do end up finding cargo-semver-checks useful and would like to see it improve further, I'd appreciate your support! I have a GitHub Sponsors page, or if it's easier for you, I'd be happy to post a bitcoin wallet address :)

Thanks for detailed version!

the pesky Halting Problem is in the way.

How is it the Halting Problem? :open_mouth: I though it's just very large set of rules that are not straightforward. Is compatibility determined by computation?

Oh, OK, we have different definitions of what is breaking change. I mean I don't disagree with yours it's just not practical in the context - I wouldn't have expected any tool like cargo-semver-checks to check behavior, just check if the code is buildable.

Of course, that makes sense. Not everyone sees things the same way, though, and given the large number of people using cargo-semver-checks, I like to be explicit about what it can and cannot promise :)

obi1:

That study is a year and a half old. cargo-semver-checks today has almost twice as many lints as it did at that time, so it's gotten much better at catching breakage since then. If we re-ran that study, we probably would find even more semver violations that we missed the first time around.

I'm pretty sure I haven't look at it in more than a couple years. So I think it's probably way more complete than I remember :).
Kix:

Yeah, I like that. We have the API break label but we need to set it before CI runs. That's probably doable for maintainers but not others. Maybe we could allow saying [API-BREAK] in commit message which sets the label before the check runs.
Also the label should invert the check so that if someone claims to do API-break but it doesn't we detect it (though maybe it'll end up as a bug report at cargo-semver-checks).

Sounds good to me. I'm not totally sure what value the label provides if the PR description or title already has [API-BREAK] in it, which we can programmatically detect just as easily as a label (I think). And we can certainly try negating the check and complaining if a claimed API break is undetected. Though my guess would be that we'll have enough false negatives that this would just need to be a warning (the bot can post a comment) rather than something that causes CI failures.

Filtering by label is super easy for instance. Warning is fine.


Hi I want to get printer data through EasyHook

Hi I want to get the data of the document being printed by the printer through EasyHook, because I want to connect to a third-party system. It prints the data directly to the paper through the printer. I want to get the data of the printer through the hook. Is there any good way?

There are a number of ways the specific app in question might be printing. You'll need to do some reverse engineering of the application itself to see what it's using. Once you figure it out, you'll want to read up on it in Microsoft's documentation to find out the API interface and how you can hook into it.


readin error for gzipped maf

I just started to use R maftools. It is a great package, however there seems to be a bug when reading gzipped maf.
For example, using this public file from TCGA https://portal.gdc.cancer.gov/files/995c0111-d90b-4140-bee7-3845436c3b42
Version: maftools_2.0.16
reading using maftools yields 75811 records
nrow(tcga.brca.somatic.maf@data)
however using fread yields 120988 records.
I have seen some SNV being assigned to wrong samples using maftools. This is a critical problem since it will affect everything.

Hi,
Thanks for using maftools and I am glad you find it useful. Regading the issue, MAF object includes only non-synonymous variants under @data slot, rest synonymous goes under @maf.silent slot. If you add up both it should be equal to your original number of rows.
><EMAIL_ADDRESS>+ nrow(brca@data)
120988

Great. Thanks.


Replacing video/audio track title

This package is amazing BTW!
Summary:
If you input the video and audio track titles as {file_name} and add a few MKV files to the source list, when you process the files the filename of the first file in the source list is used as the filename for all the tracks in all the files:
Example:

Files added to source list
"ThisMovie.mkv"
"ThatMovie.mkv"
"AnotherMovie.mkv"

Edit video and audio track name to {file_name}

Process the file

Expected result:
Examine the .mkv file with MKVToolNix - trackname should be:
"ThisMovie.mkv"
Video track name - ThisMovie
Audio track name - ThisMovie
"ThatMovie.mkv"
Video track name - ThatMovie
Audio track name - ThatMovie
"AnotherMovie.mkv"
Video track name - AnotherMovie
Audio track name - AnotherMovie
Actual Result:
"ThisMovie.mkv"
Video track name - ThisMovie
Audio track name - ThisMovie
"ThatMovie.mkv"
Video track name - ThisMovie
Audio track name - ThisMovie
"AnotherMovie.mkv"
Video track name - ThisMovie
Audio track name - ThisMovie

I'll investigate this


Default model page setup with All Regions allocation card

A basic page that has a component on it to handle adjusting water allocation for all regions - it should actually be connected so the parameter is sent to the back end and fed into the model

We don't have cards for All Regions yet, but that has another issue that can cover this


Update build-insights documentation with new modules, header unit, and PCH events

#3184

@helena-gregg : Thanks for your contribution! The author(s) have been notified to review your proposed change.

Hi @corob-msft - Should this new content be submitted in the private repo directly, or would you or someone on your team like to review it here and then move it to the private repo for quality checks and merging?
Thanks.

I am putting it in the private repo here now: https://github.com/MicrosoftDocs/cpp-docs-pr/pull/3611


Setting inputstream.adaptive.* properties vs inputstream.rtmp

Hi. Considering that is_helper.inputstream_addon could result with inputstream.rtmp (not only inputstream.adaptive), would setting inputstream.adaptive.* properties cause any conflicts in this example?
https://github.com/emilsvennesson/script.module.inputstreamhelper/wiki/Integration
playitem.setProperty('inputstream.adaptive.manifest_type', PROTOCOL)
playitem.setProperty('inputstream.adaptive.license_type', DRM)
playitem.setProperty('inputstream.adaptive.license_key', LICENSE_URL + '||R{SSM}|')

The example in the wiki is just that. If your add-on supports RTMP, the logic may be somewhat different, but that is unrelated to inputstreamhelper. ListItem properties that are not supported are ignored.

The example in the wiki is just that. If your add-on supports RTMP, the logic may be somewhat different, but that is unrelated to inputstreamhelper. ListItem properties that are not supported are ignored.

Thanks, I got it!


Update unit tests and simplify logic  for dynamic update

Russ and I worked on adding unit tests and simplify the makeright logic for dynamic update.

There is regression in the branch, withdrawing now.

I found the problem, actually it's a bug rather than a regression in mii v1 model diff code

@rjeberhard  Actually, it's not a regression, I found a bug in v1 and fixed that.  It should be good to go


Incorrect received date format?

Hi @mhale - I was just just looking through the code in relation to something and noted that you specify the received header date format as now := time.Now().Format("Mon, _2 Jan 2006 15:04:05 -0700 (MST)"). I'm pretty sure the _2 Jan is supposed to be 02 Jan else dates < 10 end up being prefixed with a space, resulting in two spaces eg: Mon,  5 Jan... rather than Mon, 05 Jan....
I could not find any exact reference in the RFC, but looking at the likes of Gmail for example they definitely prefix it with a 0 and not an additional space.
Interested in your feedback.

Hi, I've had a look and RFC 2822 Section 3.3 says the day of the month can be 1 or 2 digits, and "the numeric day-of-month MUST be between 1 and the number of days allowed for the specified month (in the specified year)". I interpret that as no leading zeroes. I agree that the extra space is less than ideal and probably breaks a parser somewhere. If you feel strongly enough about it to send a PR, I'll merge it.

I believe you're absolutely correct regarding the leading 0 - ie: not being a requirement. I actually noticed that Google use both variations within the same headers on a single message while routing within their own infrastructure (go figure). I do think the additional space (when date < 10) should be removed though, so I've submitted a PR. No rush, just good to get in there at some point. Thanks mate.


Multiple plots opened when viewing the exact same plot

Each time you click on a Plot to view the Plot, we open a new one.
Similar to https://github.com/microsoft/vscode-jupyter/issues/8591

While this is similar to the DF problem, it doesn't seem to be as bad and may be more difficult to address. Gonna just close for now.


NGP-1234: Migration scripts are not applied

Remove commands from Makefile since atlas-db requires environment variable. Follow README.md instead.

since this repo is obsolete will raise request in other repo.


Reduce exception message for loading missing files.

See #2032.
I liked @pelson's suggestion of a tick and a cross at the start of each line, but comments on this stackoverflow question suggest doing this is terminal dependent (I couldn't get the suggestions to work under my ipython anyway).  So I went for a more basic - or * bullet point difference.

The Travis failures are something to do with Dask :confused: .
FAIL: test_dask_scalar_proxy_pass_through (iris.tests.unit.lazy_data.test_as_concrete_data.Test_as_concrete_data)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/travis/miniconda/envs/test-environment/lib/python2.7/site-packages/scitools_iris-2.1.0.dev0-py2.7.egg/iris/tests/unit/lazy_data/test_as_concrete_data.py", line 105, in test_dask_scalar_proxy_pass_through
    self.assertEqual(proxy, result)
AssertionError: <iris.tests.unit.lazy_data.test_as_concrete_data.MyProxy object at 0x7f8da3bea910> != 5
----------------------------------------------------------------------```

Travis tests are failing at test_dask_scalar_proxy_pass_through.  From the comment there, I'm guessing the problem is that Travis is pointing to a later Dask version than when the test was written.

@rcomer I'm going to put up a PR to remove that test plus the changes made in https://github.com/SciTools/iris/pull/2878. Once it's merged you should rebase this PR

Thanks @djkirkham!

I did my rebase, but now Travis seems to be struggling to build anything :confused: .
conda: command not found
The command "echo 'Install Iris dependencies'; CONDA_REQS_FLAGS=""; CONDA_REQS_GROUPS="test"; if [[ "$TRAVIS_PYTHON_VERSION" == 2* ]]; then
      CONDA_REQS_FLAGS="${CONDA_REQS_FLAGS} --py2";
  fi; if [[ "$TEST_MINIMAL" != true ]]; then
      CONDA_REQS_GROUPS="${CONDA_REQS_GROUPS} all";
  fi; if [[ "${TEST_TARGET}" == 'doctest' ]]; then
      CONDA_REQS_GROUPS="${CONDA_REQS_GROUPS} docs";
  fi; CONDA_REQS_FILE=conda-requirements.txt; python requirements/gen_conda_requirements.py ${CONDA_REQS_FLAGS} --groups ${CONDA_REQS_GROUPS} > ${CONDA_REQS_FILE}; cat ${CONDA_REQS_FILE}; conda install --quiet --file ${CONDA_REQS_FILE};
  " failed and exited with 127 during .


Pl add sbmltk docs

Do those docs already exist?

I've not added docs, I've just added Readmes. I think. But maybe sth is wrong, cause the CI takes ages.
We never wrote Docs for SBMLTk, because the functionality can easily be explained in a short README.

Oh okay.


Custom component not rendering in dev mode

I have a custom components library referenced as a dependency in package.json like this:
"dependencies": {
    "nuxt": "^1.0.0-rc11",
    "vue-components": "file:../../shared/vue-components"
  }

It works well in another projects using it normally:
import { TsButton } from 'vue-components'

export default {
  name: 'app',
  components: { TsButton }
}

However, when using it in a nuxt project component renders and instantly dissapears (I guess it's rendering only in server). Throws this errors:

I've tried including it in plugins with same result.  When building and running in production mode, it works perfecly.

This may be due SSR problems with provided custom vue-components library and unfortunately, something not directly related to Nuxt.js. Would you please provide more information about that shared lib? Also, is it using named ES6 exports?

Thank you for helping.
The library actually does nothing since I'm just setting up a prototype for a new project, and I want to be sure that is doable.
Library components are generated using this and are pretty simple:
<template lang="pug">
  button.ts-button Just a testing button
</template>

<script>
export default {
  name: 'ts-button'
}
</script>

<style lang="stylus" scoped>
  .ts-button
    padding 3em
    border 20px solid black
    background-color red
</style>

And are exported with this syntax:
import TsButton from './components/ts-button'
...
export { TsButton }
...

It's weird, since:

Seems to render for an instant and then dissappears.
I can see component on node console, but is not defined on browser console.
I'm importing components in other Vue projects (no Nuxt) with Webpack with no issues.
I've tried using plugins or importing directly with same result.

If you need more info I can make a full example repo.
Thanks!

Thanks again for the info. That's strange ... Maybe sharing example repo would help a lot.

I've created a minimal sample here.
It contains:

A shared components folder, already compiled.
A simple vue spa that uses a shared component, no issues.
A nuxt starter site that uses the shared component. Component flashes in dev mode. No issues in production.

Thanks a lot for helping!


Each Sass import created seems to be a file path, not an url.

Hi, me again :)
Currently transformed imports are not in the appropriate output format.
This problem is probably only present for Windows users.
Based on sass documentation,

To ensure that stylesheets work on every operating system, Sass imports files by URL, not by file path. This means you need to use forward slashes, not backslashes, even when you’re on Windows.
ref: https://sass-lang.com/documentation/at-rules/import#finding-the-file

Here's the current behavior on Windows :
// In src/styles/main.scss
@import "abstracts/**/*.scss";

Is transformed to :
@import "C:\myproject\src\styles\abstracts\_variables.scss";

The expected behavior would be a relative path :
@import "abstracts/_variables.scss"; // or maybe @import "./abstracts/_variables.scss";

Edit :
The problem is also present on Mac.
@import "Users/myusername/Sites/myproject/abstracts/_variables.scss";

@cmalven, The solution is probably a combination of path.relative(mainSassFilePath, partialSassFilePath) and .replace(/\\/g, '/').

@maximedaoust That's very interesting. I'm not seeing that behavior on my end, on a Mac.
When I import @import 'files/**/*.scss'; the result for me is:
@import "files/_file-a.scss";
@import "files/_file-b.scss";

This is true both when running the automated test suite and when examining the processing in use on an actual site using https://github.com/antfu/vite-plugin-inspect
I trust that what you're seeing is actually occurring, but I'm not sure what would explain for the difference, at least on a Mac. I can look into this but it might take me awhile to get to it.

Thanks again for the quick reply.
Oh that's weird. Personnally, i've done my tests on Windows, but I've ask a Mac coworker to test it out just to be sure. Same behavior except the backslach.
On my end, on Windows, the problem seem to be caused by the following lines :
   // vite-plugin-sass-glob-import/dist/index.js line 65-66

   // Initially filename equals "C:/myproject/src/styles/abstracts/_variables.scss"
  filename = path.normalize(filename).replace(basePath, "");
  // After the normalize and the replace, it equal C:\myproject\src\styles\abstracts\_variables.scss
  // Normalize switch fowardslaches to backslashes and then the replace don't occur because of the backslashes
  filename = filename.replace(/^\//, "");

So here's what I would suggess to fix the issue, replace theses two lines with the following :
  filename = path.relative(basePath, filename).replace(/\\/g, '/');

Let me know if the behavior is the same as expected.

Thanks @maximedaoust that seems to work on my end. Can you please test using 1.2.1 and see if this resolves your issues?

Yeah the issue is resolved !! 🎉

@maximedaoust Woohoo! Thanks for your help.


Create analytics env vars on Azure

connecting-to-services readme describes the env vars required

Done on both staging and production


Let's include the helper template files for helm.

Fixes https://github.com/istio/istio/issues/3895

[APPROVALNOTIFIER] This PR is NOT APPROVED
This pull-request has been approved by:
To fully approve this pull request, please assign additional approvers.
We suggest the following additional approver: sebastienvas
Assign the PR to them by writing /assign @sebastienvas in a comment when ready.
The full list of commands accepted by this bot can be found here.
The pull request process is described here

Needs approval from an approver in each of these files:

release/OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

/assign @sdake

@christian-posta I don't have ability to merge to release directory although the change is valid.

/lgtm

[APPROVALNOTIFIER] This PR is NOT APPROVED
This pull-request has been approved by: sdake
To fully approve this pull request, please assign additional approvers.
We suggest the following additional approver: sebastienvas
Assign the PR to them by writing /assign @sebastienvas in a comment when ready.
The full list of commands accepted by this bot can be found here.
The pull request process is described here

Needs approval from an approver in each of these files:

release/OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

/assign @sebastienvas

/LGTM

[APPROVALNOTIFIER] This PR is APPROVED
This pull-request has been approved by: linsun, sdake
The full list of commands accepted by this bot can be found here.
The pull request process is described here

Needs approval from an approver in each of these files:

release/OWNERS [linsun]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment


persistent_datastore_pattern still required when using persistent_datastore_cluster_pattern

Describe the bug
When attempting to set both datastore_cluster_pattern and persistent_datastore_cluster_pattern (and leaving datastore_pattern and persistent_datastore_pattern undefined per the docs), I'm getting an error when attaching the persistent disk in bosh create-env.  This appears to be because the find_disk function is still hard-coded to use the persistent_datastore_pattern property:
https://github.com/cloudfoundry/bosh-vsphere-cpi-release/blob/v70/src/vsphere_cpi/lib/cloud/vsphere/resources/datacenter.rb#L127-L128
Defining the persistent_datastore_pattern to .* or any other valid pattern allows the deploy to complete.
To Reproduce
Steps to reproduce the behavior:

Convert an existing BOSH environment using the older patterns to clusters with bosh create-env

CPI Error Log
{"result":null,"error":{"type":"Unknown","message":"no implicit conversion of nil into String","ok_to_retry":false}, "log":"Rescued Unknown: no implicit conversion of nil into String. backtrace: /root/.bosh/installations/627c7175-a059-4b44-4c6e-2215b3489ebf/packages/vsphere_cpi/lib/cloud/vsphere/resources/datacenter.rb:128:in 'initialize' /root/.bosh/installations/627c7175-a059-4b44-4c6e-2215b3489ebf/packages/vsphere_cpi/lib/cloud/vsphere/resources/datacenter.rb:128:in 'new' /root/.bosh/installations/627c7175-a059-4b44-4c6e-2215b3489ebf/packages/vsphere_cpi/lib/cloud/vsphere/resources/datacenter.rb:128:in 'find_disk' /root/.bosh/installations/627c7175-a059-4b44-4c6e-2215b3489ebf/packages/vsphere_cpi/lib/cloud/vsphere/cloud.rb:610:in 'block in attach_disk' /root/.bosh/installations/627c7175-a059-4b44-4c6e-2215b3489ebf/packages/vsphere_cpi/vendor/bundle/ruby/2.6.0/gems/bosh_common-1.3262.24.0/lib/common/thread_formatter.rb:49:in 'with_thread_name' /root/.bosh/installations/627c7175-a059-4b44-4c6e-2215b3489ebf/packages/vsphere_cpi/lib/cloud/vsphere/cloud.rb:607:in 'attach_disk' /root/.bosh/installations/627c7175-a059-4b44-4c6e-2215b3489ebf/packages/ruby-2.6.5-r0.29.0/lib/ruby/2.6.0/forwardable.rb:230:in 'attach_disk' /root/.bosh/installations/627c7175-a059-4b44-4c6e-2215b3489ebf/packages/vsphere_cpi/vendor/bundle/ruby/2.6.0/gems/bosh_cpi-2.5.0/lib/bosh/cpi/cli.rb:89:in 'public_send' /root/.bosh/installations/627c7175-a059-4b44-4c6e-2215b3489ebf/packages/vsphere_cpi/vendor/bundle/ruby/2.6.0/gems/bosh_cpi-2.5.0/lib/bosh/cpi/cli.rb:89:in 'run' /root/.bosh/installations/627c7175-a059-4b44-4c6e-2215b3489ebf/packages/vsphere_cpi/bin/vsphere_cpi:72:in '<top (required)>' /root/.bosh/installations/627c7175-a059-4b44-4c6e-2215b3489ebf/packages/ruby-2.6.5-r0.29.0/lib/ruby/site_ruby/2.6.0/bundler/cli/exec.rb:63:in 'load' /root/.bosh/installations/627c7175-a059-4b44-4c6e-2215b3489ebf/packages/ruby-2.6.5-r0.29.0/lib/ruby/site_ruby/2.6.0/bundler/cli/exec.rb:63:in 'kernel_load' /root/.bosh/installations/627c7175-a059-4b44-4c6e-2215b3489ebf/packages/ruby-2.6.5-r0.29.0/lib/ruby/site_ruby/2.6.0/bundler/cli/exec.rb:28:in 'run' /root/.bosh/installations/627c7175-a059-4b44-4c6e-2215b3489ebf/packages/ruby-2.6.5-r0.29.0/lib/ruby/site_ruby/2.6.0/bundler/cli.rb:476:in 'exec' /root/.bosh/installations/627c7175-a059-4b44-4c6e-2215b3489ebf/packages/ruby-2.6.5-r0.29.0/lib/ruby/site_ruby/2.6.0/bundler/vendor/thor/lib/thor/command.rb:27:in 'run' /root/.bosh/installations/627c7175-a059-4b44-4c6e-2215b3489ebf/packages/ruby-2.6.5-r0.29.0/lib/ruby/site_ruby/2.6.0/bundler/vendor/thor/lib/thor/invocation.rb:127:in 'invoke_command' /root/.bosh/installations/627c7175-a059-4b44-4c6e-2215b3489ebf/packages/ruby-2.6.5-r0.29.0/lib/ruby/site_ruby/2.6.0/bundler/vendor/thor/lib/thor.rb:399:in 'dispatch' /root/.bosh/installations/627c7175-a059-4b44-4c6e-2215b3489ebf/packages/ruby-2.6.5-r0.29.0/lib/ruby/site_ruby/2.6.0/bundler/cli.rb:30:in 'dispatch' /root/.bosh/installations/627c7175-a059-4b44-4c6e-2215b3489ebf/packages/ruby-2.6.5-r0.29.0/lib/ruby/site_ruby/2.6.0/bundler/vendor/thor/lib/thor/base.rb:476:in 'start' /root/.bosh/installations/627c7175-a059-4b44-4c6e-2215b3489ebf/packages/ruby-2.6.5-r0.29.0/lib/ruby/site_ruby/2.6.0/bundler/cli.rb:24:in 'start' /root/.bosh/installations/627c7175-a059-4b44-4c6e-2215b3489ebf/packages/ruby-2.6.5-r0.29.0/lib/ruby/gems/2.6.0/gems/bundler-2.1.2/exe/bundle:46:in 'block in <top (required)>' /root/.bosh/installations/627c7175-a059-4b44-4c6e-2215b3489ebf/packages/ruby-2.6.5-r0.29.0/lib/ruby/site_ruby/2.6.0/bundler/friendly_errors.rb:123:in 'with_friendly_errors' /root/.bosh/installations/627c7175-a059-4b44-4c6e-2215b3489ebf/packages/ruby-2.6.5-r0.29.0/lib/ruby/gems/2.6.0/gems/bundler-2.1.2/exe/bundle:34:in '<top (required)>' /root/.bosh/installations/627c7175-a059-4b44-4c6e-2215b3489ebf/packages/ruby-2.6.5-r0.29.0/bin/bundle:23:in 'load' /root/.bosh/installations/627c7175-a059-4b44-4c6e-2215b3489ebf/packages/ruby-2.6.5-r0.29.0/bin/bundle:23:in '<main>'"}
Expected behavior
bosh create-env should run without error and attached persistent disk.
Release Version & Related Info (please complete the following information):

CPI Version:  70
BOSH Director Version:  271.11.0
Stemcell Name & Version:  bosh-stemcell-1.33-vsphere-esxi-ubuntu-bionic-go_agent.tgz
vCenter Version:  6.7
NSX(T/V) Version (If using):  N/A

Thanks @amhuber this definitely looks like a bug that should be addressed. While you are waiting for us to correct it, I think as a workaround you can just put an unmatchable regex into the persistent_datastore_pattern (e.g. "^$") and it should start working for you. Provided that the value isn't nil, the code should carry on and find your datastore in one of the accessible clusters.

Yep, I mentioned in the issue that it works with any valid pattern.  I'm using .* right now, and using ^$ would be equivalent (searching all datastores), it would just do it in the "other" code block.

Yeah, agreed this isn't urgent.  To be honest, if you just update the docs to indicate you need to use ".*" for the persistent_datastore_pattern when using the cluster options you could just leave it at that.

I looked at all the places where those values were used and the cluster settings would take precedence if they were configured, other than the code mentioned in this issue that was specifically for finding a detached persistent disk.  In that code, if the configured regex doesn't match (for example by using ^$), then it will fall back to searching all datastores anyway.  Given that, I couldn't find anything that would work differently with .* or ^$.
If you are aware of a location where the non-cluster regex will override the cluster regex, then I'd appreciate it if you could point out the code.

Ah, thanks, I missed that OR in the logic, that will definitely cause an issue.  I'll use ^$ until this is fixed.

closing and reopening to make a new tracker story


DBZ-7302 Implement Snapshotter SPI Oracle

closes: https://issues.redhat.com/browse/DBZ-7302
The PR is based on https://github.com/debezium/debezium/pull/5196, must be rebased after its merge.

[x]  Implement the Snapshotter interface
[x]  Implement the SnapshotLock interface
[x]  Implement the SnapshotQuery interface
[x]  Add test for a custom Snapshotter

/packit test --labels oracle

/packit test --labels oracle

/packit test --labels oracle

Since supporting all possible snapshot modes for the currently refactored connectors (PostgreSQL, MySQL, Oracle) is taking more time then expected, I am tracking the changes in this https://github.com/debezium/debezium/pull/5245

After the review this PR can be merged separately

/packit test --labels oracle

Failures seem not related.

/packit test --labels oracle

/packit test --labels oracle

/packit test --labels oracle

@jpechane Do you want to wait fixes on async engine before merging this?

@mfvitale Yes, let's stabilkez the testusite first so we have again valid results.

/packit test --labels oracle

@jpechane Failures are not related. I have tracked some flaky tests with https://github.com/debezium/debezium/pull/5245/commits/321273cdb295c098e2d35c1cb3695251e33fedd0 and https://github.com/debezium/debezium/pull/5245/commits/901b2bf7ced3068f5b8d73b83391f7e7a059f84c on the other PR. Do you think we can merge this?

@mfvitale Applied, thanks


Remove useless sempahore object

Very small PR, just removing an orphaned semaphore.
I see you've been working on the socket handling. Good job again.
I believe if we want to further improve network throughput on server we should change how we read and write sockets. Most people will agree that the read() approach is sub-optimal.
There are a couple of solutions, like SocketAsyncEventArgs and RIO, but maybe using DotNetty could settle the issue. What do you think?

I don't want to introduce 3rd party libraries in this library. So DotNetty is not an option for me.
I saw SocketAsyncEventArgs  but never go into detail. Do you have some? What do you mean with RIO?
But is there a more lower API than writing to a Networkstream?

I also found a solution for the TimeoutAfter code. Please have a look and let me know what do you think.

RIO is aka Registered I/O http://www.serverframework.com/asynchronousevents/2011/10/windows-8-registered-io-networking-extensions.html
But it's windows only, so it's a no go as well. I also came to known that .net sockets are migrating to SocketAsyncEventArgs  under the hood already on dotnet core.
The great challenge on server applications is to move data quickly from a lot of sockets into the application code. Queuing tasks will end up in a lot of context switches. Libraries like Netty or ZeroMQ usually address that problem by pooling multiple sockets on a single thread, thus reducing context switching. DotNetty is extensively used  inside Microsoft and is the base networking layer for their IoT hub for example, I think it's worth a try. Or maybe even as a separate package that can be plugged in, optionally by users. It's most useful on the server.
I guess the most obvious step right now is optimize the packet parsing. I'm writing a proposal using a state machine and will let you evaluate if it's a good approach.

Not submitting a PR, very experimental, but check my branch https://github.com/israellot/MQTTnet/tree/optmizations
The center idea is instead of reading from socket stream byte by byte, have a state machine where I can pull as many bytes as I could read and it will dispatch callbacks when a complete packet has been read. Partial data is kept on state, so we can always read the stream fully.
By doing so we can avoid a lot of context switching, specially for small packets that will arrive in a single tcp packet. It also enables us to have a push pattern for received messages, which also avoids overhead of having a lot of tasks waiting for input. Tasks are dispatched when input is available.
If you find the idea interesting, we can move forward. Need proper unit testing, etc.

Meanwhile, made a little amend to my serializer changes. Array.Resize would allocate another buffer to copy the previous one.
It probably doesn't impact much, but feels better.

From code perspective your solution looks good but from performance view I have doubts. I already tested a prototype which loads always 4KB from the underlying stream and returns the data from a cache. So if the cache has data no stream is accessed. But this leads to -5.000 messages per second in my usual tests. The overhead in managing this cache was stil too much. So I habe doubts if managing "steps" etc. will decrease or increase performance.

On my tests i got a  bit of increase in performance. But I see it as a first step towards reducing GC. I am concerned about the number of ephemeral buffers and objects created in the heap that are garbage collected. I think we should also create more realistic tests, with more parallel message producers and listeners clients, which is the more real use case of mqtt.
About the cache, I’ve looked at the implementation for sockets on dotnet core and they did a good job on the internals so a cache in front of it wouldn’t really help much indeed. On core 2.1 things will get even better, to a point that it will be replacing libuv on kestrel, so we should be able to steal some ideas from there on how they architectured sockets.
Anyway it has been fun to work and experiment on this library. Thanks for the constant feedback


GPG - sign & verify pax files

zopen-build can now generate signed builds, and zopen-install can verify the signature before installation.
We can enable signed builds using the --sign-pax or -sp option. If this option is set, it reads the ZOPEN_GPG_SECRET_KEY_FILE and its passphrase from GPG_SECRET_KEY_PASSPHRASE to sign the file.
These are mandatory for signing a file.
The public key is read from GPG_PUBLIC_KEY_PATH. The public key and the signed file details are written to metadata.json (this JSON is not part of the PAX).
Since this is a major change, we have increased the metadata.json file version to 0.2.
Further, during installation, the signed file will be verified. We have a --skip-verify option that can be used if the metadata.json version is less than0.2 Otherwise, the verification of signed PAX files is mandatory

Hi @IgorTodorovskiIBM Thank you for quick review. I have addressed all your comments.

/run tests

Hi @IgorTodorovskiIBM @DevonianTeuchter @MikeFultonDev  - I have addressed the first round review comments. I have created a sample pipeline to generate signed build and i have also manually uploaded these signed builds to install & verify. Request your reviews to proceed on this further. Thank you.

@IgorTodorovskiIBM Thank you for the review. I have addressed all the comments. Please review.


fix warp in directory issue

Describe the pull request
before fix:
when warp in directory is true, it will generate  srcdir="${tmpdir}/${NAME}" rm -rf "${srcdir}" when warp in directory is false, it will also generate  srcdir="${tmpdir}/${NAME}" rm -rf "${srcdir}"

nice catch, thank you!

released https://github.com/kamilsk/godownloader/releases/tag/v0.1.2


chore(rest): add openapi spec to the tree

Add pre-commit hook to make sure we update the spec as we change the
rest code.
Add another step to the CI linter to check that the spec is up to date.
(Note the linter here isn't quite right for this)

bors try

bors try

bors merge

bors merge

bors merge


1.10: Changes for LDB section on the form

Change the last point in instructions to
"All personal information is collected in accordance with section 26(c) of the Freedom of Information and Protection of Privacy Act (FOIPPA) for the purposes identified above. Any questions or concerns regarding the collection of your information may be directed to by submitting a request to<EMAIL_ADDRESS>
~2. Change "Your government email address" to "Your email address" in Written Complaint section
(Question: Are there any constraints on the email?)~

Make "Your Ministry" optional in Written Complaint Section

Change "Government email address:" to "Email address" in Recipient of written complaint section

Change  Public Service Agency (PSA) to Liquor Distribution Board (LDB) in the Next Steps section

Feedback document attached for reference
https://app.zenhub.com/files/439477060/2684f101-ea4d-4291-8473-ea010fc52d97/download

On point 5, it should read Liquor Distribution Branch, not Liquor Distribution Board.

@Andrew-Vargas-bcgov to change the fullform

Updated<EMAIL_ADDRESS>to be an e-mail link.
The e-mail field gets updated to display "Your preferred email" when the user selects that they are LDB.
Field is optional.
Updated to "Recipient's email address"
Changed all instances of "Liquor Distribution Board" to "Liquor Distribution Branch".

Andrew,
This all looks good to me, with two exceptions:

If I first select LDB, then check “not LDB” when opening the form, it leaves “Liquor Distribution Branch” as the default ministry/organization.  Once I checked “not-LDB”, it should clear that field?
·        When I identify as being from LDB, in the first bullet of “Before you submit” the text reads:
o    “When you submit your complaint, it will be automatically sent to you, the recipient and to the LDB Employee Relations Department, Public Service Agency (PSA), who will provide a copy to your union within 10 days”
o    It should read: “When you submit your complaint, it will be automatically sent to you, the recipient and to the LDB Employee Relations Department, Liquor Distribution Branch (LDB), who will provide a copy to your union within 10 days”  [changes highlighted]
Don Smallwood, Director of Investigations & Intake
Employee Relations Division | BC Public Service Agency
167 Lorne Street | Kamloops BC | V2C 1V9
Office:<PHONE_NUMBER> | Mobile:<PHONE_NUMBER>
Living with gratitude and respect on the unceded territory of the Splatsin

From: Andrew Vargas @.>
Sent: January 17, 2023 2:14 PM
To: bcgov/digital-journeys @.>
Cc: Smallwood, Don PSA:EX @.>; Comment @.>
Subject: Re: [bcgov/digital-journeys] 1.10: Changes for LDB section on the form (Issue #839)
[EXTERNAL] This email came from an external source. Only open attachments or links that you are expecting from a known sender.

Updated<EMAIL_ADDRESS>to be an e-mail link.
The e-mail field gets updated to display "Your preferred email" when the user selects that they are LDB.
Field is optional.
Updated to "Recipient's email address"
Changed all instances of "Liquor Distribution Board" to "Liquor Distribution Branch".

—
Reply to this email directly, view it on GitHubhttps://can01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fbcgov%2Fdigital-journeys%2Fissues%2F839%23issuecomment-1386153705&data=05|01|Don.Smallwood%40gov.bc.ca|17dbda41736841bbe58c08daf8d83330|6fdb52003d0d4a8ab036d3685e359adc|0|0|638095904692816138|Unknown|TWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D|3000|||&sdata=dOyzvZ2QUggZXw3nycIny2fnDKV1uU%2BrHWiy4Cml%2BNA%3D&reserved=0, or unsubscribehttps://can01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FA37NTNJLAPK6I5ULK4A5EWTWS4KUHANCNFSM6AAAAAATSZGHWU&data=05|01|Don.Smallwood%40gov.bc.ca|17dbda41736841bbe58c08daf8d83330|6fdb52003d0d4a8ab036d3685e359adc|0|0|638095904692816138|Unknown|TWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D|3000|||&sdata=89gZgfTRIam0tZKz%2F7yl2viEdxi%2F%2BwfGRWoGtrM%2Fz7M%3D&reserved=0.
You are receiving this because you commented.Message ID: @.***>


(MightyPen, 2019/June/03)
Specifically, the proposed new sentence says:
"new_reseed_value will not be used as the identity if the insertion of a row into the table was attempted in a rolled back transaction, even if the table contains no rows after the roll back."
'pmasl' suggestion (of Apr 29) might be an iterative improvement over the particular phrasing of the above proposed new sentence.

@ispedals : Thanks for your contribution! The author, @pmasl, has been notified to review your proposed change.

The note may seem confusing. Suggest changing to:
|DBCC CHECKIDENT ( table_name, RESEED, new_reseed_value )|Current identity value is set to the new_reseed_value. If no rows have been inserted into the table since the table was created, or if all rows have been removed by using the TRUNCATE TABLE statement, the first row inserted after you run DBCC CHECKIDENT uses new_reseed_value as the identity.If rows are present in the table, or if all rows have been removed by using the DELETE statement, the next row inserted uses new_reseed_value + the current increment value.If a transaction inserts a row and is later rolled back, the next row inserted uses new_reseed_value + the current increment value as if the row had been deleted. If the table is not empty, setting the identity value to a number less than the maximum value in the identity column can result in one of the following conditions: -If a PRIMARY KEY or UNIQUE constraint exists on the identity column, error message 2627 will be generated on later insert operations into the table because the generated identity value will conflict with existing values. -If a PRIMARY KEY or UNIQUE constraint does not exist, later insert operations will result in duplicate identity values.|

This public PR 2030 successfully brought to our attention a need for improvement in the DBCC CHECKIDENT documentation.
It was necessary to use another PR to adjust the phrasing of the new sentence (https://github.com/Potapy4/sql-docs/pull/160).
Therefore, this PR 2030 does not need to be Merged now, and indeed would Conflict with the re-phrased update.
I am now Closing this successful PR 2030.
@ispedals Thank you very much for bringing this documentation shortcoming to our attention.


Use DIDComm v2

Will need to upgrade the pico engine to version 1.3.x

Will need to replace index.html, plan.html, and about.html

Will need to install the io.picolabs.did-o (which is installed in all newly created picos in v 1.3.x+) in existing picos
Will need an application to make connections

These things have been addressed.

Manually did the upgrade and replaced the HTML files
Manual installation of the did-o ruleset will be needed for older affiliate picos
The io.picolabs.plan.connect ruleset (which uses oob as a module) now exists

Operational note: notify all earlier affiliates and explain how to install io.picolabs.did-o in the Technical section of the Manage applications page.


Implement auth

Can can we query private spreadsheets?

(1) Create service account (see instructions).

(2) Download key as JSON.
(3) Manage API client access at https://admin.google.com/${DOMAIN}/AdminHome?chromeless=1#OGX:ManageOauthClients

Add the "Unique ID" from step 1 as the "Client Name"
Add https://www.googleapis.com/auth/spreadsheets.readonly as the scope.

(4) Make sure to impersonate a user, otherwise it will fail:
from __future__ import print_function
from google.oauth2 import service_account
from googleapiclient.discovery import build
from httplib2 import Http
from oauth2client import file, client, tools

# If modifying these scopes, delete the file token.json.
SCOPES = ['https://www.googleapis.com/auth/spreadsheets.readonly']

def main():
    """Shows basic usage of the Sheets API.
    Prints values from a sample spreadsheet.
    """
    SERVICE_ACCOUNT_FILE = '/Users/bdealmeida/Downloads/gsheetsdb-2bb6767cc084.json'
    creds = service_account.Credentials.from_service_account_file(
        SERVICE_ACCOUNT_FILE, scopes=SCOPES)
    delegated_creds =<EMAIL_ADDRESS>    service = build('sheets', 'v4', credentials=delegated_creds)

    # Call the Sheets API
    SPREADSHEET_ID = '1hsxoXlOTQzMgNdS4Hp6tg-nVRq8v4Mc7syYZsOHltTk'
    RANGE_NAME = 'A2:B3'
    result = service.spreadsheets().values().get(spreadsheetId=SPREADSHEET_ID,
                                                range=RANGE_NAME).execute()
    values = result.get('values', [])

    if not values:
        print('No data found.')
    else:
        print('Name, Major:')
        for row in values:
            # Print columns A and E, which correspond to indices 0 and 4.
            print(row)

if __name__ == '__main__':
    main()

Based on:

https://developers.google.com/sheets/api/quickstart/python
https://developers.google.com/api-client-library/python/auth/service-accounts

Scope should be https://spreadsheets.google.com/feeds  instead.

^ https://developers.google.com/chart/interactive/docs/spreadsheets