qsctl support rsync

rsync is a popular utility that is used for synchronizing files.
https://en.wikipedia.org/wiki/Rsync
I'm wondering if we can extend qsctl to synchronize files from a remote server which supports rsync protocol.
One scenario would be: linux repository synchronization

a remote server here means what ? QingStor Server or just a Linux server with rsyncd ?
For QingStor Server
It's obvious that qingstor will not support rsyncd for now, so qsctl can't support this.
For Linux server with rsyncd
qsctl is a unix like tools for user to use qingstor, andrsync is not the thing that qsctl should do. And you can use rsync directly.
To summarise, we will not support this feature.

I mean the user can sync files from file server which supports rsync protocol without downloading all of the artifacts. And rsync can work with ssh to copy files between two linux hosts. It is a powerful utility and is widely used in unix world.  I think it is a de-fecto standard way to sync files between unix hosts.

I did agree with you that rsync is very uesful, but it doesn't mean we should do it by qsctl.
First of all, qsctl just did work between QingStor Server and Local, and rsync is not in the map.
Then, qsctl is tool for all platform, we should also take windows, darwin into consideration.
In a word, I don't think we should do this job in qsctl, and I will be happy to discuss how to sync files in other ways.

rsync works under windows. And adopting rsync doesn't mean just invoking rsync directly. There are a few rsync protocol implementations available.  And I think this utility should support different datasource because the general process of moving data from/to qingstor can be reused although there are a lot of implementation details in different protocols.

Closed for not implement in qsctl and use another method instead.


pleaser support snappy uncompress

OS
Mac
VERSION
1.6.1
ISSUE DESCRIPTION
We use snappy to compress and uncompress the value, please support snappy! I try to use a custom formatter, but it doesn't work!
I go build https://github.com/golang/snappy to generate exec. And use /user_path/snappy -d {VALUE} or -d {HEX}, but they doesn't work!

please have a look! thanks.

The hex representation needs to be converted into bytes and then passed to snappy. You can't pass {HEX} directly.

The hex representation needs to be converted into bytes and then passed to snappy. You can't pass {HEX} directly.

Thanks to reply. I tried {VALUE}, it didn't work either. Can you tell me how to do? thanks.

The bytes will be passed like this: c0ed09.... I implemented support for LZ4 in C# like this:
using System.Text;
using K4os.Compression.LZ4;

// this converts the hex into bytes two chars at a time
byte[] bs =
  Enumerable.Range(0, args[0].Length)
    .Where(x => x % 2 == 0)
    .Select(x => Convert.ToByte(args[0].Substring(x, 2), 16))
    .ToArray();
string result = Encoding.UTF8.GetString(LZ4Pickler.Unpickle(bs));
Console.WriteLine(result);

The bytes will be passed like this: c0ed09.... I implemented support for LZ4 in C# like this:

completely right, @Hawken94  the value passed to your script is hex value, you should convert it to bytes first


Search client samples fixes

There were a couple of samples that missed updates to use new subclients. These are not shipped in the package so any users should see the correct versions as soon as this is merged.

cc @rakshith91 just an FYI that I went ahead and took care of this


Please help me restore.

Describe the issue
I need help restoring documents to basic default in this platform, but unsure how to  reinstate. I private carrier network isn't able to operate. Device stating no internet connection, but I'm clearly connected. How to to fix my location and use to device? Not receiving calls or text messages from anyone I previously interacted with. Did it block my string
Code of Conduct

[X] I agree to follow the IETF's Code of Conduct

How do I restore endpoint


Fix FunctionTypeProcessor registration

Fix FunctionTypeProcessor registration issue

@oclay1st Please sign the Contributor License Agreement!
Click here to manually synchronize the status of this Pull Request.
See the FAQ for frequently asked questions.

@oclay1st Thank you for signing the Contributor License Agreement!

@oclay1st thank you for making your first contribution to Spring Native.


Fork, Commit, Merge - Easy Issue (NextJS)

Fork, Commit, Merge - Easy Issue (NextJS)
Create a sample product list with names and prices
Note: You don't have ask permission to start solving the issue or get assigned, since these issues are supposed to be always open for new contributors. The actions-user bot will reset the file back to previous state for the next contributor after your commit is merged. So you can just simply start working with the issue right away!
Description:

Your task  is to create at least three sample products inside of the sampleProducts array with names and prices for landing page of an imaginary online store. The landing page displays a welcome message, a brief description of the store, and the sample products.

Start by navigating inside of the directory used in this project with the command:
cd tasks/nextjs/easy/nextjs-easy`

Then run npm i to install NextJS, React and all the necessary dependencies for the project.
After installing the dependencies you can open the index.tsx file inside of src/pages directory, and start implementing your solution!

This project uses Tailwind for CSS and all the classes are already added to the components.
This project also uses TypeScript but you don't need to know any TypeScript to work with this issue. As soon as you add the sample products with names and prices, TypeScript will infer the types and all the errors present in the file will go away.
After you are done with the list, make sure you are in the tasks/react/easy/next-easy in your terminal and run npm run dev to start the NextJS server. Then, open "http://localhost:3000/" in your browser to check how the landing page looks.
If the product names and prices are showing up as they should, you are ready to make a pull request!

To work with this issue, you need to have npm and NodeJS installed to your local machine.
Check out README.md for more instructions of installing npm and NodeJS as well as how to make a pull request.

Feel free to ask any questions here if you have some problems!
Also, kindly give this project a star to enhance its visibility for new developers!

Hi nikohoffren, not sure if this is a typo?

make sure you are in the "tasks/nextjs/easy/nextjs-easy" in your terminal instead of "tasks/react/easy/next-easy"

Hi nikohoffren, not sure if this is a typo?

make sure you are in the "tasks/nextjs/easy/nextjs-easy" in your terminal instead of "tasks/react/easy/next-easy"

Good call! That is indeed a typo, it should be tasks/nextjs/easy/nextjs-easy because NextJS is in its own directory even though it is a React framework.


Revert emmc support

Revert "Add eMMC driver support"
Also revert a newer commit to resolve conflict

The CI error is something we have inherited from upstream and we get the error in all builds


Feature #146/rename directory and files

closes #146
Note that when deleting a file the folder wont switch to another file or show an empty editor.
Will make another issue for this.

Issue already exists #250

to


Update zio-sbt-website Plugin

@googley42  Do you know why it gets stuck when running sbt update:
milad@nixos ~/s/s/zio-dynamodb (update-zio-sbt-website-plugin) [1]> sbt update
[info] welcome to sbt 1.4.7 (Oracle Corporation Java 11.0.15)
[info] loading settings for project global-plugins from idea.sbt ...
[info] loading global plugins from /home/milad/.sbt/1.0/plugins
[info] loading settings for project zio-dynamodb-build from plugins.sbt ...
[info] loading project definition from /home/milad/sources/scala/zio-dynamodb/project
[info] loading settings for project root from build.sbt ...
[info] set current project to root (in build file:/home/milad/sources/scala/zio-dynamodb/)
[info] Updating
https://repo1.maven.org/maven2/com/amazonaws/DynamoDBLocal/1.17.0/DynamoDBLocal-1.17.0.pom
  | => root / update 71s
  | => zio-dynamodb / update 71s


[ALBERT] albert-xlarge V2 seems to have a different behavior than the other models

Hi, this issue is related to ALBERT and especially the V2 models, specifically the xlarge version 2.
TLDR: The ALBERT-xlarge V2 model seems to be different to other V2/V1 models.
The models are accessible through the HUB; in order to inspect them I save the checkpoints which I then load in the modeling.AlbertModel available in the modeling.py file. I use this script to save the checkpoint to a file.
In a different script, I load the checkpoint in a model from modeling.py (a line has to be added to the scope so that the modeling scope begins with module, same as the HUB module). I load the checkpoint in this script. In that same script I load a HUB module, and I compare the outputs of both models given the same input values.
For every model, I check that the difference is near to zero by checking the maximum difference between tensor values (included at the bottom of the second script).
I have had no problems replicating the SQuAD results on all of the V1 models, but I could not do so on the V2 models apart for the base one. Is this related? Thank you for your time.

(a line has to be added to the scope so that the modeling scope begins with module, same as the HUB module).

Hi @LysandreJik
I have tried to add module scope, but I don't seem to get it working with a line.
Below is how I get the module scope done and get your compare_albert.py working, doesn't look good but it works.
Could you tell how you get module scope done?
Thank you,
insop@neon ~/Downloads $ diff -uN a.py b.py
--- a.py        2019-11-25 01:07:50.000000000 -0800
+++ b.py        2019-11-25 01:08:23.000000000 -0800
@@ -1,4 +1,4 @@
-def get_assignment_map_from_checkpoint(tvars, init_checkpoint, num_of_group=0):
+def get_assignment_map_from_checkpoint(tvars, init_checkpoint, num_of_group=0, add_scope='module'):
   """Compute the union of the current variables and checkpoint variables."""
   assignment_map = {}
   initialized_variable_names = {}
@@ -8,8 +8,15 @@
     name = var.name
     m = re.match("^(.*):\\d+$", name)
     if m is not None:
-      name = m.group(1)
-    name_to_variable[name] = var
+      # add 'module' scope name to match tf hub module
+      if add_scope is not None:
+          name = 'module/' + m.group(1)
+      else:
+          name = m.group(1)
+      # NOTE: store name as value for scope matching
+      # since 'var' value was not used
+      name_to_variable[name] = m.group(1)
+
   init_vars = tf.train.list_variables(init_checkpoint)
   init_vars_name = [name for (name, _) in init_vars]

@@ -20,7 +27,7 @@
   else:
     assignment_map = collections.OrderedDict()

-  for name in name_to_variable:
+  for name, old_name in name_to_variable.items():
     if name in init_vars_name:
       tvar_name = name
     elif (re.sub(r"/group_\d+/", "/group_0/",
@@ -50,7 +57,12 @@
       if not group_matched:
         assignment_map[0][tvar_name] = name
     else:
-      assignment_map[tvar_name] = name
+      if add_scope is not None:
+        # add 'module' scope name to match tf hub module
+        # <'module/'+ xxx, xxx>
+        assignment_map[tvar_name] = old_name
+      else:
+        assignment_map[tvar_name] = name
     initialized_variable_names[name] = 1
     initialized_variable_names[six.ensure_str(name) + ":0"] = 1

Hi @insop, to add the module scope, I added the following line at line 194 of modeling.py:
with tf.variable_scope("module"):

Which results in the __init__ method of AlbertModel beginning with these few lines:
[...]
    config = copy.deepcopy(config)
    if not is_training:
      config.hidden_dropout_prob = 0.0
      config.attention_probs_dropout_prob = 0.0

    input_shape = get_shape_list(input_ids, expected_rank=2)
    batch_size = input_shape[0]
    seq_length = input_shape[1]

    if input_mask is None:
      input_mask = tf.ones(shape=[batch_size, seq_length], dtype=tf.int32)

    if token_type_ids is None:
      token_type_ids = tf.zeros(shape=[batch_size, seq_length], dtype=tf.int32)

    with tf.variable_scope("module"):
      with tf.variable_scope(scope, default_name="bert"):
        with tf.variable_scope("embeddings"):
          # Perform embedding lookup on the word ids.
          (self.word_embedding_output,
[...]

Hi @LysandreJik
Thanks a lot, it works like a charm!

Great to hear, please let me know if you manage to convert the v2 models/reproduce the results!

Hi @LysandreJik
I have ran your script (compare_albert.py) for v2 models.
My run large model shows more difference, not as large as your data for xlarge.
For xlarge, difference seems okay.
I have a question to run squad, but I will post in other open issue link that I saw you were there.


import/order crashes for typescript files, if no package.json file exists since version 2.23

While verifying a typescript file, the exception below is thrown for the import/order rule, if no package.json file exists. There is no such problem with version 2.22
Oops! Something went wrong! :(
ESLint: 7.27.0
TypeError [ERR_INVALID_ARG_TYPE]: The "path" argument must be of type string. Received null
at new NodeError (node:internal/errors:329:5)
at validateString (node:internal/validators:129:11)
at dirname (node:path:1172:5)
at getFilePackagePath (/home/jpalm/webdev/3rdpartylibs/node/node_modules/eslint-plugin-import/lib/core/packagePath.js:15:558)
at getContextPackagePath (/home/jpalm/webdev/3rdpartylibs/node/node_modules/eslint-plugin-import/lib/core/packagePath.js:15:383)
at typeTest (/home/jpalm/webdev/3rdpartylibs/node/node_modules/eslint-plugin-import/lib/core/importType.js:109:2291)
at resolveImportType (/home/jpalm/webdev/3rdpartylibs/node/node_modules/eslint-plugin-import/lib/core/importType.js:109:2724)
at computeRank (/home/jpalm/webdev/3rdpartylibs/node/node_modules/eslint-plugin-import/lib/rules/order.js:319:40)
at registerNode (/home/jpalm/webdev/3rdpartylibs/node/node_modules/eslint-plugin-import/lib/rules/order.js:335:16)
at handleImports (/home/jpalm/webdev/3rdpartylibs/node/node_modules/eslint-plugin-import/lib/rules/order.js:617:11)

Oops, I just saw now that there is already this issue here: https://github.com/benmosher/eslint-plugin-import/issues/2096


CORE-20434: Remove crypto tenant

Crypto tenant was hardcoded in CryptoProcessor which meant cluster db entity manager was always returned.
This PR splits the getEntityManager into two, one for vNode and one for clusterDb.
We now get the clusterDbEntityManager directly in CryptoProcessor.

We are going for a different approach, where splitting entity manager factory won't be necessary.


HuangcHengshun

Welcome to your shiny new Codespace running React! We've got everything fired up and running for you to explore React.
You've got a blank canvas to work on from a git perspective as well. There's a single initial commit with the what you're seeing right now - where you go from here is up to you!
Everything you do here is contained within this one codespace. There is no repository on GitHub yet. If and when you’re ready you can click "Publish Branch" and we’ll create your repository and push up your project. If you were just exploring then and have no further need for this code then you can simply delete your codespace and it's gone forever.
This project was bootstrapped for you with Vite.
Available Scripts

Al：ProTip! Add comments to specific lines under Files changed.commit into github:main from huangchengshun:patch-1
Conversation 1
Commits 1
Checks 0
Files changed 2
Conversation
huangchengshun
@huangchengshun huangchengshun commented 12 hours ago
Welcome to your shiny new Codespace running React! We've got everything fired up and running for you to explore React.
You've got a blank canvas to work on from a git perspective as well. There's a single initial commit with the what you're seeing right now - where you go from here is up to you!
Everything you do here is contained within this one codespace. There is no repository on GitHub yet. If and when you’re ready you can click "Publish Branch" and we’ll create your repository and push up your project. If you were just exploring then and have no further need for this code then you can simply delete your codespace and it's gone forever.
This project was bootstrapped for you with Vite.
Available Scripts
@huangchengshun
HuangcHengshun
7b556cc
huangchengshun commented 11 hours ago
IMG_20240316_062830
@joshaber joshaber closed this 6 hours ago
Closed with unmerged commits
This pull request is closed, but the huangchengshun:patch-1 branch has unmerged commits. You can delete this branch if you wish.
If you wish, you can also delete this fork of github/codespaces-react in the settings.
Add a comment
Comment
ProTip! Add comments to specific lines under Files changed.
Something went wrong. Please try again.
Remember, contributions to this repository should follow its contributing guidelines, security policy, and code of conduct.
ProTip! Add comments to specific lines under Files changed.
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development
Successfully merging this pull request may close these issues.
None yetcommit into github:main from huangchengshun:patch-1
Conversation 1
Commits 1
Checks 0
Files changed 2
Conversation
huangchengshun
@huangchengshun huangchengshun commented 12 hours ago
Welcome to your shiny new Codespace running React! We've got everything fired up and running for you to explore React.
You've got a blank canvas to work on from a git perspective as well. There's a single initial commit with the what you're seeing right now - where you go from here is up to you!
Everything you do here is contained within this one codespace. There is no repository on GitHub yet. If and when you’re ready you can click "Publish Branch" and we’ll create your repository and push up your project. If you were just exploring then and have no further need for this code then you can simply delete your codespace and it's gone forever.
This project was bootstrapped for you with Vite.
Available Scripts
@huangchengshun
HuangcHengshun
7b556cc
huangchengshun commented 11 hours ago
IMG_20240316_062830
@joshaber joshaber closed this 6 hours ago
Closed with unmerged commits
This pull request is closed, but the huangchengshun:patch-1 branch has unmerged commits. You can delete this branch if you wish.
If you wish, you can also delete this fork of github/codespaces-react in the settings.
Add a comment
Comment
ProTip! Add comments to specific lines under Files changed.
Something went wrong. Please try again.
Remember, contributions to this repository should follow its contributing guidelines, security policy, and code of conduct.
ProTip! Add comments to specific lines under Files changed.
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development
Successfully merging this pull request may close these issues.
None yet


Select component

There is a requirement to provide a select component within an application.
The select will act like a regular form Select component, though allows the developer to specify both regular text content or html content.
The code for a select should specify a list of options, each one specifying a value and a text label or children. The current value and field name must be provided to and when a value changes it called onChange with a regular field change type event.
Finally the component must work as a Formik component.
Below are examples of use, note the version to select fleet and how the badge is aligned.

The custom Select component
The custom Select component is a turbo-charged version of the native <select> input. Instead  of relying on the browser to handle displaying the <option>'s, this Select displays a custom sheet that can be enhanced with additional components.
Component States
The Select has eight variations, depending where it currently is in its lifecycle:

To begin, the Select looks like a regular input, with the label at full height.
Hovering on the Select will add a 2px blue (0 spread) box-shadow to the input.
Clicking on the Select will expand the list:

Firstly, the Select container will animate to a height of 5 items (or less). Anything else will be hidden behind a scroll.
Each Item will then fade in, from top to bottom.

Hovering on any of the Items will add a light grey background to it.
Clicking on any item will cause the container to collapse back to its original height, with the new selected item displaying.

When an item has been selected, the Select works much the same, except now:

The Select label is now small and showing the previously selected value
When the list is expanded, the currently selected item will have a blue background in the list.

Scrolling
If a Select contains over 5 items, then scrolling should be added. When a user scrolls the list, a subtle shadow is added to the main area of the Select. The items will then disappear under it.

Notes

I think we'll need to discuss the different types of content that will be required for the custom Select. The Select has been sized according to inputs, allowing them to sit flush with one another in a form based layout - this gives us a good approximation of the size that the additional content should be. Overloading a custom Select with a lot of content feels like it would be overly cumbersome for the user to interact with, and may be better served via the (proposed in JIRA) Carousel component.

Changes from the original design

Reduced the overall size of the badges to ensure they aren't increasing the height of the selects to be larger than their input counterparts.
Fixed the alignment of the badge with the text label.
Removed the horizontal dividing lines between each item.
Added a Hover & Active states to the Select.
Added an Active state to the already selected item.
Added a Hover state to the items.

Any chance of a Sketch file to go with this or something we can inspect?

I've exported the component into its own Sketch file to keep it temporarily separate from the main Standards Toolkit file:
CustomSelect.zip

I agree the component shouldn't be over used but think the custom select is a big improvement on default select, especially having the Blue Highlight in the Active with selection and Hover with selection states. Is there any reason for not having one select component with all of the features of the custom select?

@hxltrhuxze
It seems a little strange to me how the active and hover highlights overlay the scrollbar.

@m7kvqbe1 Yeah it's a fair point - I was following OSX standards of having scrollbars appear on scroll. If we're having permanent scroll bars, then I agree we should move the highlight so it isn't clipping.
On the subject of scrollbars though, will we get stuck with the native behaviour? Checking the Can I Use support for styling scrollbars and it's still behind flags/not supported in the lowest browser we need to support

Can we avoid overriding the browsers rendering of scroll bars and just set it to that the scroll is generated by the browser/os if it's needed or not at all if not

I have a work in progress version of this in a project. Once we have the design signed off I'll complete it and get it into the common library

looks good from a design perspective.

Done some investigation and thinking back to how I've used Select components before. The existing design documented above doesn't take into consideration the requirement to be able to type into a select field and get an autocomplete like experience. This is important for some applications that have a long list of options or people who need to use the keyboard for navigation.
Add to this there are some times when the option list needs to be fetched from the server as the user types if there are many many options. Finally the lookup of options as the user types needs to allow for custom filtering logic in case the information displayed contains other information in it too that should not be considered in the regular filtering. Will investigate using a 3rd party component for this and customising it for use in the component kit and ensure it works with Formik as selects and autocomplete components can be alot of hard work.


[CT-417] [Bug] User experiencing dbt deps loop when trying to import private package

Is there an existing issue for this?

[X] I have searched the existing issues

Current Behavior
User is attempting to import package via a HTTPS and user/PAT. Once dbt deps is run, it looks like there's a loop even when the repo has been cloned.
19:36:15  Set downloads directory='/tmp/xxxxx'
19:36:15  Executing "git clone --depth 1<EMAIL_ADDRESS>xxxxxxxx"
19:36:16  STDOUT: "b''"
19:36:16  STDERR: "b"Cloning into 'xxxxxxxx'...\n""
19:36:16  Pulling new dependency xxxxxxxx.
19:36:16  Executing "git rev-parse HEAD"
19:36:16  STDOUT: "b'xxxxxxxx\n'"
19:36:16  STDERR: "b''"
19:36:16    Checking out revision master.
19:36:16  Executing "git remote set-branches origin master"
19:36:16  STDOUT: "b''"
19:36:16  STDERR: "b''"
19:36:16  Executing "git fetch origin --depth 1 --tags master"
19:36:16  STDOUT: "b''"
19:36:16  STDERR: "b'From https://xx.com/xx/template\n * branch            master     -> FETCH_HEAD\n * [new tag]         1.0        -> 1.0\n'"
19:36:16  Executing "git tag --list"
19:36:16  STDOUT: "b'1.0\n'"
19:36:16  STDERR: "b''"
19:36:16  Executing "git reset --hard origin/master"
19:36:16  STDOUT: "b'HEAD is now at xxxxxx Update packages.yml\n'"
19:36:16  STDERR: "b''"
19:36:16  Executing "git rev-parse HEAD"
19:36:16  STDOUT: "b'xxxxxx\n'"
19:36:16  STDERR: "b''"
19:36:16    Checked out at xxxxxx.

Then attempts to reclone:
19:36:16  Executing "git clone --depth 1<EMAIL_ADDRESS>xxxxxx"
19:36:16  STDOUT: "b''"
19:36:16  STDERR: "b"fatal: destination path 'xxxxxx' already exists and is not an empty directory.\n""
19:36:16  command return code=128
19:36:16  Updating existing dependency xxxxxx.
19:36:16  Executing "git rev-parse HEAD"
19:36:16  STDOUT: "b'xxxxxx\n'"
19:36:16  STDERR: "b''"
19:36:16    Checking out revision master.
19:36:16  Executing "git remote set-branches origin master"
19:36:16  STDOUT: "b''"
19:36:16  STDERR: "b''"
19:36:16  Executing "git fetch origin --depth 1 --tags master"
19:36:17  STDOUT: "b''"
19:36:17  STDERR: "b'From https://xx.com/xx/template\n * branch            master     -> FETCH_HEAD\n'"
19:36:17  Executing "git tag --list"
19:36:17  STDOUT: "b'1.0\n'"
19:36:17  STDERR: "b''"
19:36:17  Executing "git reset --hard origin/master"
19:36:17  STDOUT: "b'HEAD is now at xxxxxx Update packages.yml\n'"
19:36:17  STDERR: "b''"
19:36:17  Executing "git rev-parse HEAD"
19:36:17  STDOUT: "b'xxxxxx\n'"
19:36:17  STDERR: "b''"
19:36:17  Already at xxxxxx, nothing to do.

Expected Behavior
dbt deps should be able to clone the package into the dbt_packages folder successfully.
Steps To Reproduce

Create PAT for user.
Create packages.yml with private repo HTTPS and user:PAT (PAT can be saved in env_var, also happens with hardcoded)
Run dbt deps.

Relevant log output
No response
Environment
- dbt: 1.0.4

What database are you using dbt with?
bigquery
Additional Context
No response

Hi @crystalro0, sorry to hear you're having issues with dbt deps.
To help me investigate, could you post the output when you run the same command with json log output? i.e. dbt --log-format=json deps. This will help me identify exactly which code paths you're hitting in dbt-core.

The method producing those log messages is definitely clone_and_checkout:
https://github.com/dbt-labs/dbt-core/blob/5071b00baac3ff2a9bcce565d09661605de52654/core/dbt/clients/git.py#L132-L174
The first time, it completes successfully and fires GitProgressCheckedOutAt. In subsequent tries, it finds that the package is already installed, and ultimately fires GitNothingToDo. The question is, why is it running clone_and_checkout over and over again?

@crystalro0, @jtcohen6 was just able to reproduce this behavior on a call with me just now by creating a package that depends on itself. Check this package and see if there might be any circular references causing this loop.

closing in favor of #4975


ros2 v0.8.0 roi_image_saver

[x] package.xml

[x] CMakeLists.txt

[x] node info

[x] Build pass
Starting >>> roi_image_saver
Finished <<< roi_image_saver [11.3s]

[x] Test Pass Summary: 4 tests, 0 errors, 0 failures, 0 skipped


Add disableMachineServiceAccount configuration to GCE

What this PR does / why we need it:
This PR adds a flag to the GCE provider named disableMachineServiceAccount. If true, it does not add a Service Account to the Machine, leaving it empty.
In a lot of use cases, GCE nodes will not need to interact with the Google APIs, so they can safely not have a Service Account.
In the future, if needed, we could add another flag to specify a customMachineServiceAccount, but that's not my use case for now.
Which issue(s) this PR fixes (optional, in fixes #<issue number> format, will close the issue(s) when PR gets merged):
Fixes #1194.
Optional Release Note:

NONE.

/ok-to-test

Thanks for the great work.

This doesn't fix https://github.com/kubermatic/machine-controller/issues/1194 so this shouldn't be linked with that issue IMHO.
Please add a release note. Could be Add disableMachineServiceAccount configuration to disable the usage of service account for GCE machines
Please take of linting issues from pull-machine-controller-lint

hey @ahmedwaleedmalik, I did all the changes... the pull-machine-controller-lint was complaining about the monitoring package change, I had already it rolled back anyway.
requesting a re-review :)

/retest

@xrstf can you please overridepull-machine-controller-e2e-vsphere. It's failing due to the docker hub rate limits.

/override pull-machine-controller-e2e-vsphere

Thanks so much!
Also, do you know when a release including this would be made?

@jonatasbaldin we are planning on having a new release in the coming week.

Perfect, thanks :)

Awesome, thanks!


New files are listed as existing with 1 diff

Hello,
Several new files still not existing in the target system show listed as existing with 1 diff:

And if I see the diff... it clearly shows that it does not exist:

Cheers,
Nuno

issue reproduced, only happens when files are hidden,
Hiding files:

Showing files:

Great that you managed to reproduce it.
So I'll show the files for now then.
Actually I much prefer to hide them since in ABAP you don't really need to know the underlying files, just the objects, so thank you for the option to hide them which I only recently found :)
Cheers,
Nuno

:smile:
I guess the fix for showing "new @remote" vs "1 diff" when hiding files is pretty simple, however I do not have time to look into it this week or next.
https://sessioncatalog.sapevents.com/go/agendabuilder.sessions/?l=133&sid=37856_464065&locale=en_US

Wow! Congratulations!
Great to know. Everyone should be aware of what you're accomplishing.
On Thu, Nov 3, 2016 at 11:22 AM, Lars Hvam<EMAIL_ADDRESS>wrote:

😄
I guess the fix for showing "new @remote https://github.com/remote" vs
"1 diff" when hiding files is pretty simple, however I do not have time to
look into it this week or next.
https://sessioncatalog.sapevents.com/go/agendabuilder.sessions/?l=133&
sid=37856_464065&locale=en_US
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/larshp/abapGit/issues/407#issuecomment-258117614, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AAgeUfmLErgfhiGNLe9OnLVTHckj88x2ks5q6cQRgaJpZM4Knk-2
.

Guys, are you sure about the direction of solution ?
"New" is a diff in fact. I'd rather fix the diff page to show new files as well as changed.
What I was thinking about recently is that this "new" marker should actually be replaced with "diff" for all cases, However there should be some marker in the line showing the state of changes in local and remote. Because it can also be both remote and local change ! It can be more convenient to observe. Something like this:

Can fix whet I have time. I have currently a go-live in my project so quite busy :(

@sbcgua what you suggest makes a lot of sense. I fully subscribe that approach.

sounds good to me :smile:

Started looking into this topic. Several questions.

Opinion needed :) What is this "new" state ? Is it needed (valuable) at all ? I mean e.g. "local new" can be also described as "changes to commit", and "remote new" as "changes to pull". And with "changes" I mean not only new changes but any changes. Does it look logical ? Is specific "new" marker give anything useful? Asking because it affects how to design the markers - separate "new" and "changes" gives too many variants which might be less clear.

Code questions @larshp

lcl_file_status=>status currently returns new local objects with MATCH = 'X'. Is it a bug or intended ? It is not always but just for completely new objects. Looks like a bug but maybe used somewhere.

Existing obj with 1 new file

Completely new obj

The same method returns empty filename for new local objects. Although ! The lt_local contains filename, so can be copied to the output results if required. This blocks diff preview for local new files to be displayed. I wonder if empty filename is used anyhow as a trigger somewhere ? And can I adjust it (fill filenames for new local  objs)

1: I think it adds value to show the "new" states. It is more informative than number of diffs/changes
2: not sure :smile: just try changing it if needed, and perhaps add some unit tests
3: yeah, just try
My thoughts: if all lines for a object has NEW = 'L', then the object is "new @local", guess something similar can be done for "new @remote"

@larshp: typically, started as an easy task but not that easy in reality :)
One of issues: there is not enough information now to say if the file was changes locally or remotely. Local file can be compared to local checksum from DB, but there is no info for remote. Probably remote check sum should be saved as well. And here is a question: as I understood the local checksum is calculated just on file data, whereas git checksum includes also git blob type and file size. Why ? Was there any intention to simplify local checksums ? Could be good if unified imho. What do you think ?

git checksum includes the blob type, unifying is a good idea

@larshp Have a look. If OK, I'll PR - this is an independent code in fact. update_checksums is not actually needed, it is more for current tests so can be removed later.

Almost done, have a look (better click) :)
w files

wo files

all diffs are shown now (new files and also in stage page)
modified status is shown independently for local and remote (compared to last sync state)
internal: file_status rewritten to support all of that, improved performance and hopefully structure + a UT

Still to finish one thing - push/pull should not update all local state but only actually changed - looks easy at the first glance but I have a feeling ...

cool, what does the red "~" mean? Is this for deletions? And will it be possible to determine if an object is deleted locally? hmmm, perhaps a feature to add later

~ is for "mixed" - a group of files that have both "added" and "modified". It has a tooltip also. But seems that it is not obvious from design =( I'll think how to make it clearer.
deletion would be a nice feature definitely :) Actually it is easy to add now (a marker). I can even do from UI perspective. But the the object deletion itself of course is a separate topic.

That works with older IEs although, for old GUI with IE7 will not work

Final look (support deleted also)

Still to finish one thing - push/pull should not update all local state but only actually changed - looks easy at the first glance but I have a feeling ...

This appeared to be tricky indeed. Will PR after finish this but not sure when

perhaps change "X" to "D"eleted or "R"emoved?

Ok, so finally the code is coming. Here is a final final look:

I've put some change comments to PR and also to the new changelog file.
The stuff is definitely worth testing (I did how I could :). It should work fine but ... there are lots of changes.
P.S. At the beginning of the first start it will check local checksums and propose to update them. This should be safe unless you have both remote and local changes. If you do - better sync repos before the abapGit update. If you did update already ... well ... the best idea is probably to copy older abapGit and sync repos with it first. Or carefully push local changes you know to remote individually and then "Update local checksums" in advanced menu. The idea is to have changes just on one side, then "Update checksums" should produce proper result.
P.P.S There might be some ghostly bug which I cannot recreate for the moment. Sometimes some files are shown as MM (both side changes) although changed just from one side. Seems like this happens because local state in DB looses one of files. This also triggers Stage and Pull showing at the same time. If you find why or how to recreate - pls report - I cannot find it for the moment and seen it just once. Maybe no bug at all, just dev state - but I pushed some changes, then merged master branch to feature branch (outside of abapGit) and then abapGit showed MM for one file (zabapgit.pro.abap) - it was just missing from DB state for some reason.

OK, mentioned bug seems to be resolved (hopefully)
Plus, added global view diff to the main page

Plus, added change-side marker to diff page and now if changes are remote colors are shown vice versa

One issue. lcl_tadir=>get_object_package does not work reliably for deleted objects. delflag seems to be not enough. As I understand it is set when transport is released ... not sure ... I don't know this stuff. Do you know a reliable way to check if the object is deleted (even if it is not yet released) ? I tried with DTELs - they don't mark delflag =( Programs seem to do ... completely confusing.

Methods lif_object~exists are used to determine if the object exists or not,
https://github.com/larshp/abapGit/blob/master/src/zabapgit_object_dtel.prog.abap#L78

@nununo can you try the latest version of abapGit, it has a lot of changes regarding showing diffs, thanks

@larshp I am getting ready to try it. But as far as I could understand the repositories must be previously sync'd before I do the upgrade, right? I read it here:

P.S. At the beginning of the first start it will check local checksums and propose to update them. This should be safe unless you have both remote and local changes. If you do - better sync repos before the abapGit update. If you did update already ... well ... the best idea is probably to copy older abapGit and sync repos with it first.

So, just to be on the safe side, I'm syncing everything before doing the upgrade.
I'll probably upgrade today and provide feedback.
Thank you guys for the speed at which this is evolving!

@larshp, now I sync'd all the code. But I'm still worried about this:

This should be safe unless you have both remote and local changes.

Since my repositories insist in keeping small diffs... due to different kinds of flags and small details in the generated code... we can say that they are in a state where you have both remote and local changes, right?
What's the risk of upgrading now?

Worst case abapGit will overwrite local changes with the remote object when doing a pull. (however currently there is a bug which prevents this)
This is the same behavior as earlier abapGit versions, so I guess you have experienced this earlier?

Upgraded. All went smoothly. Great new features! Thanks and congratulations!

:tada:
There are a few bugs in the new version, follow https://github.com/larshp/abapGit/issues I will try to fix most of them sometime this week

I'm closing the issue, a lot has happened since the issue was raised.
Just open a new issue if the issue is still there

@nununo Glad that it looks useful :)

I'm intrigued about one thing in this new feature.

I changed something in the source system.
So in the target system abapGit showed: _M
Then I manually replicated the change directly in the target system
And now abapGit showed: MM (locally changed and remotely changed)
But when doing a DIFF it doesn't show any differences
Only after the PULL did the object stopped showing as having differences

Is this the expected normal behavior now?
Not that it is a problem. It's ok and actually interesting :)
Thanks,
Nuno

Yes, that is intended behavior ... although I also had a doubt about it and maybe it should be adjusted. Here is my logic:

There are 3 states: actual local (1), actual remote (2) and state at the moment of last sync (3).
To identify changes abapGit compares 1 to 3 and 2 to 3. (3) is stored in DB as a hash (because otherwise the whole file content should be stored ... maybe in future #357 )
consequently even though (1) = (2) in your case - it is not similar to (3)

IMHO the fact that files were changed is more important than that they are equal. But that's arguable - I just miss additional arguments :) Maybe there can be some special status for this case ... maybe 2 Ms but in green ... but would not it look overcomplicated ... so here I need some external opinions :)

I think the way it is now makes sense :)
Maybe... I'd just move those to the end of the list of modified files... since they're in sync anyway and having them at the end would allow us to focus on the ones which really are different. But this is just a detail borderlining on luxury :)

Mmm... thinking about additional state. Smth like this (~)

But really not sure - states will become ambiguous ...
@larshp what do you think ?

not sure, no opinion so far


[DON'T MERGE] Support classical exit fee

Support classical exit fee

Codecov Report

Merging #173 (c8e4792) into develop (426234d) will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff            @@
##           develop     #173   +/-   ##
========================================
  Coverage    71.73%   71.73%
========================================
  Files           68       68
  Lines         2257     2257
  Branches       335      335
========================================
  Hits          1619     1619
  Misses         638      638

Flag
Coverage Δ

batch-submitter
59.59% <ø> (ø)

contracts
89.52% <ø> (ø)

core-utils
54.48% <ø> (ø)

data-transport-layer
37.68% <ø> (ø)

message-relayer
71.69% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 426234d...c8e4792. Read the comment docs.


worm face gears

would be very useful!
example:
https://ntrs.nasa.gov/api/citations/20020092092/downloads/20020092092.pdf

What is the benefit of this compared to a hypoid bevel gear?   Basically, why are these specifically useful?
This looks interesting and if there's a good case for it then it could happen eventually, but at the moment, basic worm gears are broken and we don't know how to fix them, so this isn't going to happen soon.   I skimmed over the attached document and it is very technical.  I couldn't tell if it clearly lays out how to construct the form for these gears.

What is the benefit of this compared to a hypoid bevel gear? Basically, why are these specifically useful?

just higher gear ratios AFAICT.  bummer on worm gears being broken.


BUG: Top of the dialog pop-up unclickable when overlapping with menu bar

The dialog pop-up, such as for "Create new event", has the top bar (with the name and the "x" to close it) which becomes opaque and unclickable when it is dragged to the menu bar (where Schedule-It Home, Calendar, Manage Events, Past Meetings, Logout are).

It has been fixed by using the collapsible menu bar which Steven implemented (week 6)


Add a tool to suggest increasing the severity of performance-impacting bugs

Resolves #1709
Autonag wiki page
{{AutonagRule
  | Rule name = Severity of performance impacting bugs
  | Purpose   = Consistency and help getting traction on bugs with a high impact on performance
  | Action    = Needinfo the assignee or the triage owner (if not assigned) to increase the severity
  | Source    = severity_high_performance_impact.py
}}

Checklist

[ ] Type annotations added to new functions
[ ] Docs added to functions touched in main classes
[x] Dry-run produced the expected results
[ ] The to-be-announced tag added if this is worth announcing

Dry-run
2022-11-24 20:56:10,145 - INFO - Run tool severity_high_performance_impact.py
2022-11-24 20:56:13,959 - INFO - Tool severity_high_performance_impact.py has finished.

The following bugs have discrepancy between the bug severity and the performance impact evaluation. Owners got needinfo requests to bump the severity:

BugSummarySeverity

1789268

DrawSurfaceWithShadow in canvas is very slow

S3

1771902

High CPU use to make cursors blink in google docs in fully occluded windows

S3

1761734

Slow drawing of this PDF file

S3

1760045

Slow HTMLElement.focus calls on matrix-react benchmark

S3

1755864

Process using 100% of the CPU on the main thread recording infinite recursion error stacks

S4

1752841

Pretty bad Google Docs perf with GPU canvas.

S3

1745512

Significant performance impact when using Webpagetest and ES Modules

S3

1741865

Firefox loads tabs slowly after starting and update to new version

S3

1725781

SAP application very slow since 72.0b1, with deeply nested percent-height tables

S3

1694048

Reddit Become Slower

S3

1686592

Responsiveness: On Twitter, switching from Home to Explore takes 2.5x as long as in Chrome

S3

1677509

SendReceiveMouseInputEvent often blocks the main thread of the parent process

S3

1664492

Extremely long DNS resolution times with Fenix

S3

1647153

Slow reloading of Pocket Explore on Moto G (XT1031)

S3

1602842

OSXDisplay::EnableVsync hangs in CVDisplayLinkCreateWithActiveCGDisplays

S3

1595680

Slightly different values in display lists causing extra picture cache invalidations.

S3

1550754

ContentParent::KillHard janks the main thread

S3

1389712

switching to a Google Slides tab is slow (long repaint)

S3

1363383

Channel switching on Slack is slow

S3

1356686

OMT brotli decompression

S3

Comment example - bug 1752841
The severity field for this bug is set to S3. However, the Performance Impact field flags this bug as having a high impact on the performance.
:lsalzman, could you consider increasing the severity of this performance-impacting bug?
For more information, please visit auto_nag documentation.


add reflex perf mode

add rx.Config (perf_mode) or env variable (REFLEX_PERF_MODE). Value should be an enum with warn, raise and ignore. Default should be warn, to avoid breaking changes.
warn: Warn about possible performance implications
raise: raise Exception if performance issues are found
ignore: do nothing, disable checks completely
iirc reflex currently only checks for state sizes. I would love to add another check for always dirty substates (caused by uncached vars).

TODO: add socketio limit https://github.com/reflex-dev/reflex/pull/4392/files#r1851155673


Add annotations to support TLS

This one is interesting because it has bigger implications for how scraping works.
It might be as easy as setting __scheme__ to https, which could be done with an annotation like k8s.grafana.com/metrics.scheme.
However, if it's an https target, we'd need to probably set the TLS cert.
Easy approach would be to set __scheme__ and see what happens in the prometheus.scrape component.
Medium approach would be to set __scheme__ in the initial discovery.relabel, then use two more discovery.relabel components to include and exclude https schemes, then use two prometheus.scrape components.

First step would be to identify a workload that shares metrics via TLS and without other auth (which we wont support initially).

A few of the control plane components use https (controller manager, scheduler, api server), e.g.:
  prometheus.scrape "controllermanager" {
    job_name   = "kube-controller-manager"
    targets    = discovery.relabel.controllermanager.output
    scheme     = "https"
    bearer_token_file = "/var/run/secrets/kubernetes.io/serviceaccount/token"
    tls_config {
      insecure_skip_verify = true
    }
    clustering {
      enabled = true
    }
    forward_to = [prometheus.relabel.controllermanager.receiver]
  }

I can see agent has picked up on the annotations after the following command on the kubeadm-controlplane cluster:
kubectl -n kube-system annotate pods kube-controller-manager-ip-172-31-16-35 \
  k8s.grafana.com/scrape="true" \
  k8s.grafana.com/job="annotate-controllermanager" \
  k8s.grafana.com/metrics.portNumber="10257" \
  k8s.grafana.com/metrics.scheme="https"

Here's what the agent sees on discovery.relabel.annotation_autodiscovery_pods:
__meta_kubernetes_namespace                                                = "kube-system",
__meta_kubernetes_pod_annotation_k8s_grafana_com_job                       = "annotate-controllermanager",
__meta_kubernetes_pod_annotation_k8s_grafana_com_metrics_portNumber        = "10257",
__meta_kubernetes_pod_annotation_k8s_grafana_com_metrics_scheme            = "https",
__meta_kubernetes_pod_annotation_k8s_grafana_com_scrape                    = "true",

Not seeing metrics or any scrape debug info though.

The new helm test will run a config analysis pod that'll easily print out the discovery and scrape components. You can then see if there were failures in scraping.

I ran the test but can't find a way to see the output:
helm test grafana-k8s-monitoring
NAME: grafana-k8s-monitoring
LAST DEPLOYED: Fri Dec 15 18:06:33 2023
NAMESPACE: default
STATUS: deployed
REVISION: 1
TEST SUITE:     test-grafana-k8s-monitoring
Last Started:   Mon Dec 18 13:38:28 2023
Last Completed: Mon Dec 18 13:38:28 2023
Phase:          Succeeded
TEST SUITE:     analyze-grafana-k8s-monitoring
Last Started:   Mon Dec 18 13:38:28 2023
Last Completed: Mon Dec 18 13:38:36 2023
Phase:          Succeeded
TEST SUITE:     test-grafana-k8s-monitoring
Last Started:   Mon Dec 18 13:38:36 2023
Last Completed: Mon Dec 18 13:38:42 2023
Phase:          Succeeded


[Maps] Add interface for setting collection locations after uploading CSVs

Description

This PR adds an interface for setting collection locations after uploading CSVs. We agreed to invest in this (with automatic geosearching) before officially announcing the maps feature.
I synced up with @happyimadesignr on the design and interactions but she's welcome to review more.

Notes

MetadataCSVLocationsMenu: new component that shows up under the CSV box. We'll automatically geosearch for what the user has in their CSV. The results show up in the menu, and they can fix them from there.
MetadataInput: Renamed 'locationWarning' to just 'warning'. Moved processLocationSelection into GeoSearchInputBox for modularity.
MetadataUpload: Added step for kicking off geosearchCSVLocations and added MetadataCSVLocationsMenu below all the issues based on Mark's recommendations. Added metadata as a prop to access metadata in this component.
LocationsController: Added a results limit option for efficiency when we only want 1 geosearch result.
Added a LOCATION_UNRESOLVED_WARNING.

Demo

Tests

Go through the flows shown above at http://localhost:3000/samples/upload

Nice!!!

Could we change "Location Matches" to "Confirm your Collection Locations"? Slightly clearer title.
Also, for the warnings such as "unresolved plain text", "changed to state", can we do yellow warning text with a small alert icon? Similar to (without the border):

I wonder if it would make sense to wrap the locations menu in something like this:

Except light blue, with the "Confirm your Collection Locations" title outside (similar to Warnings and Errors).
The reason for this is that then, all the different elements follow a consistent styling. Right now, you have the collapsible warnings/errors, but then the locations menu is styled differently. If you have both warnings and also the csv location menu, I think the inconsistent styling will look a bit jarring.
Can you try that and see how it looks?

Those are good ideas Mark, to wrap in in the same notification background in light blue and the input warning style. Your title suggestion is nice too, more clear.
Other than that, can we move the location column closer to the sample name? And still keep the table full width.
I will mock it up!

What happens when there is no match? Adds free text?

Yes, it "stays" as free text.

Do we have plans to remove free text? It is a way to circumvent our privacy controls.

No current plans AFAIK. We've discussed this before but we can talk about it more in some docs. I added some more notes to the mapping doc: https://czi.quip.com/gzWdAafIX8Yb/LocationMapping-Sprint-Plan

Could we change "Location Matches" to "Confirm your Collection Locations"? Slightly clearer title.
Also, for the warnings such as "unresolved plain text", "changed to state", can we do yellow warning text with a small alert icon? Similar to (without the border):

Done and done 👍

(altho I just realized you didn't ask for the border :o)

@MarkAZhang  @jshoe
Here's what I was imaging from our convo:

Turned it into an IssueGroup, as discussed ;)


feat(MatRipple): The style of the ripple component is still in the M2 state, whether to consider upgrading to M3?

Feature Description
Hi all, is there any plan to upgrade the styling of MatRipple components to M3?
Use Case
No response

My understanding is that the ripple is the same between M2 and M3. Can you elaborate on what changes you're expecting?

My understanding is that the ripple is the same between M2 and M3. Can you elaborate on what changes you're expecting?

Hello, it's a pleasure to talk to you.
Preface
As far as I know, the ripple effect of M2 has no blur effect, while the ripple in M3 should have a blur effect.
You can refer to ripple in this website, (@material/web is published by Google's material-components team). Ripple in the @material/web project is implemented like this
Compare
For comparison, you can try opening these two websites for comparison:

M2 Ripple
M3 Ripple


Update dependencies and workflows

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 100.00%. Comparing base (afcfb86) to head (5be74ae).

Additional details and impacted files
@@            Coverage Diff            @@
##              main      #107   +/-   ##
=========================================
  Coverage   100.00%   100.00%
=========================================
  Files           34        34
  Lines          764       764
  Branches        33        33
=========================================
  Hits           764       764

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


Random page number access for read flow

The people want to be able to go directly to a certain page when reading the Haggadah.

TODO

[ ] Page selector
[ ] Center the paging elements


Date Output Not Import Friendly

For my purposes, the date format from JSON 2020-06-25T15:32:32.510000Z is not so friendly.
I'm not sure what the ideal format for most folks would be, but for me only the date is needed so I used:
=MID(C2,6,2) & "/" & MID(C2,9,2) & "/" & LEFT(C2,4) in my spreadsheet program to turn the date string into a "MM/DD/YYYY" string. Probably some python to reformat as part of the extraction program would be better.
This works nicely for my tax purposes.

Thank you for all the feedback and code, I will incorporate them as soon as I get some time.
Out of necessity, do you use any tool for wash sale calculations? Is there an analyzer software or excel? I dont trust RH one bit .


Query helpers

Adds get_unique and get_unique_mut to extend the query api and cover a common use case. Also establishes a second impl block where non-core APIs that don't access the internal fields of queries can live.

I ran into another really useful application for this today: unit testing and debugging with marker components.
When unit testing (or interactively debugging), you'll often set up toy examples where you create a few entities, give them each a different marker component, and then query them to inspect their behavior. This pattern is made much more ergonomic with get_unique and get_unique_mut.
The other approaches don't work well here:

Refactoring to use a resource instead of a singleton component. This doesn't work at all, since the point is to test out your existing systems without modifications.
Storing the Entity ID within a Resource, then fetching that resource, and using query.get(my_entity). This is an added layer of indirection for now real benefit, and makes your tests much less clear. In general, this pattern also has serious issues with ArchetypeComponent access clashes compared to just using a marker component.
Manually reimplementing this functionality. Very tedious boilerplate when writing many tests; especially since you want nice error messages if something breaks.

@cart I think this is good to merge, if you've been convinced that it's a useful API :)

Another really central pattern that's coming up for my turn-based RPG:

Only one unit can act at once, since it's turn-based.
I very much want to be able to have each unit as an entity, and their data as components, rather than sticking them into resources, for what should be obvious reasons.
In order to determine which unit is active, I give them a temporary  Active (or Attacker etc.) marker component.
Then, I need to extract the other components for that entity, querying only for the Active entity. These queries will always return exactly one entity, is a logic error if it doesn't, and the entity in question changes over time.

This is a clear, idiomatic use of get_unique in real game code, and isn't appropriately handled by either converting it to a resource or storing the Entity in a resource and then looking things up within the system using query.get.

Regarding naming, I think "unique" is unfortunate. In many contexts, "unique" is understood as de-duplication, i.e. removing duplicate elements. Instead, I would suggest "single" for this functionality, as it clarifies you're dealing with a single element and is more common:

Rust single crate
Kotlin single() and singleOrNull() functions

Could thus be get_single() and get_single_mut() or also just simply single() and single_mut().

Regarding semantics, I think cart's point is important here:

Yeah this is sort of the crux of my hesitance to adopt get_unique(). It encourages people to design around "single entity" patterns. Maybe thats a good thing, but its a paradigm shift with implications we need to consider.

I've only started to use Bevy recently, but it seems to me that its components are specifically designed and optimized for being used across a large number of entities, and there are already different approaches for the problems stated here:

single-entity components which model a singleton (one global instance) can be represented as resources
in alice-i-cecile's use case, it might be more straightforward to store a entity_in_turn: Entity variable and use World::get() to access it, rather than shifting components around

Could thus be get_single() and get_single_mut() or also just simply single() and single_mut().

I'm alright with that. The idea of "there should only be one" is conveyed a bit more clearly with "unique" over "single" however.

single-entity components which model a singleton (one global instance) can be represented as resources

This doesn't work for complex singletons whose fields you want to be able to mutate in parallel. It also fails badly when you have common behavior that you would like to share with other entities. This is a common pattern for really simple games like roguelikes and platformers, and I'd hate to either push people into making their singleton players resources, iterating over a query that they know is always length one or rolling their own version of this constantly.
Re the paradigm shift discussed by @cart: pushing people towards using complex resources in Bevy is commonly a trap: it strips away a huge amount of the modular, expressive power of the ECS and tends to block parallelism. Singleton entities are superior to resources for everything that isn't a simple natural monolith.

in alice-i-cecile's use case, it might be more straightforward to store a entity_in_turn: Entity variable and use World::get() to access it, rather than shifting components around

FWIW, you'll almost never want to use World::get directly, since it requires a completely-blocking exclusive system :) Using query::get instead has some serious issues though still:

It's much less clear what's happening from the system's type signature or function body.
There's dramatically more boilerplate within the function body that can be messed up.
You have to store the data nonlocally, which makes it harder to reason about.
It's very painful to extend this pattern to more-than-one entity of this sort. Marker components scale naturally; you just replace the internal system logic with a loop rather than needing to change the entire piping around them.
You get mutable access to every entity in the entire query, despite only needing to modify a tiny bit of its data.
This can create spurious system ambiguity issues that make it harder to spot real issues, since the ambiguity checker can't see past the type signature.

With the addition of sparse set components, marker components will become even faster, and I would expect to benchmark much better than the Resource approach, in addition to having better conceptual clarity.

Yes, i am strongly in favor of encouraging "singleton entities" over resources, because they get many advantages due to being made of components. We should not encourage the liberal use of resources.
For example, a game might only have one player. But some components of that player (like health) might be common with other entities. It's nice to have systems that operate on all those entities (say, health system), while also having systems that operate specifically on the player.
This is the idiomatic and preferable way to do things as it is, regardless of whether this new API is merged or not.
The big selling point of this new API is that it makes these use cases safer. It gives you a way to actually be explicit about your intentions/invariants and enforce them. Because there are many use cases in typical games where you'd only have one of a particular entity.
It doesn't make sense to think of "but i want to leave the possibility of there being more than one player at some point", if you know very well that that will never be the case in your game.

@alice-i-cecile @jamadazi Thanks a lot for the clarification! That sounds like single() might be a replacement for resources, or at least for many of the cases where resources are used today.
From an API design perspective and Bevy's aspiration to be simple, I believe it would make sense if there are some clear guidelines on when to use which approach (Query::single() vs. Query::get() vs. resources), ideally with straightforward examples. It would also allow to clearly see what the benefits of multiple coexisting APIs are.

Yes, as I was saying, it is a matter of teaching. We can teach users to have the proper understanding.
The API is useful and idiomatic IMO, so it should be accepted. Even with the concern that it might make things slightly more confusing to beginners.
We already have people (notably alice and myself) who are actively teaching newcomers good practices and mentality around resources vs entities, and other ECS patterns. We'll just adapt to this.

I think if this gets adapted to the latest changes on main, I think this is good to merge.

Cool to see this moving ahead! 🙂
Maybe concerning the name of the methods, just so this isn't overlooked:

Regarding naming, I think "unique" is unfortunate. In many contexts, "unique" is understood as de-duplication, i.e. removing duplicate elements. Instead, I would suggest "single" for this functionality, as it clarifies you're dealing with a single element and is more common:

Rust single crate
Kotlin single() and singleOrNull() functions

Could thus be get_single() and get_single_mut() or also just simply single() and single_mut().

If the consensus is that "unique" is a better term than "single", that's OK!

Regarding semantics, is there a guideline/rule of thumb when to use resources vs. this pattern?
I have quite some code based on resources, when should I consider moving to this instead?

Maybe concerning the name of the methods, just so this isn't overlooked:

Ah yeah sorry for not commenting here (overlooked is the right word). I don't have strong opinions here. std has managed to avoid usage of both single and unique terms, which is amazing to me. So drawing from the consensus across other languages / ecosystems is a good call. I'll add C# to the stack of things that uses "single" in this context (returns the next item in an enumerator if it is the only value in the enumerator, otherwise throws an exception).
My vote is for single unless counterexamples are provided from other prominent projects. I do also like the shorter single() over get_single().

Regarding semantics, is there a guideline/rule of thumb when to use resources vs. this pattern?
I have quite some code based on resources, when should I consider moving to this instead?

This is still TBD, but i think its fair to say that Resources are for "unique" types that do not have an "identity" other than their type and should not be "correlated" to other values/types. They are "singletons" for things like Asset collections, Time, etc.
query.single() / query.unique() is for things that need an identity other than their type and might correlate to other things (ex: a "player" might have a Name, a Velocity, a Position, etc).
In my mind nothing has really changed here in the Entity vs Resource relationship other than the fact that you can now add a "single" assertion to entity access.

My vote is for single unless counterexamples are provided from other prominent projects. I do also like the shorter single() over get_single().

FWIW, I like get_* and get_*_mut over just single because what you get out of the function perfectly parallels the existing Query::get and Query::get_mut APIs. Rather than passing in a specific entity to look for, you're just grabbing the only entity possible.

My justification for the "inconsistency" is that get(thing) terminology is generally used when retrieving key->value pairs. Whereas terms like next(), unwrap(), and first() are for "operations that return a thing".

Could .only() work?

My justification for the "inconsistency" is that get(thing) terminology is generally used when retrieving key->value pairs. Whereas terms like next(), unwrap(), and first() are for "operations that return a thing". single() seems to fit cleanly into the latter group.

This makes sense. .only() seems like the best option I've seen.

Thanks for all the feedback! 🙂
I tend to agree with cart that it's a special method very similar to first() and last() and a get_ prefix is rather unusual.
only() would again have the problem that it's not common (in existing crates or other languages).
Additionally, "only" also means "just/merely" in English, while "single" is a property implying 1 element.

I really don't like only. It throws me off, looks unusual.
single is my favorite, i strongly prefer it over unique
i like the get_ prefix, but it's not a strong opinion, it's fine without the prefix, too

@cart, there is a lot of opinions here. If you can make a final call, I'd like to finalize this PR.

I still prefer single() so lets roll with that. I think it reads better and has the benefit of consistency in the wider language ecosystem.

bors r+

@cart I found the ecosystem equivalent of this method: exactly_one. Might be worth renaming and unifying behavior?

ecosystem equivalent

Is this used outside of itertools? If it is not widely prevalent, IDK if we could call it that, and might not be worthwhile to copy. I personally haven't seen it anywhere.
single still feels more intuitive to me.


Uncaught TypeError: Cannot read property 'substring' of undefined

[Enter steps to reproduce below:]

Just searched for references (keyboard shortcut) on an empty file

Atom Version: 1.8.0
System: Mac OS X 10.10.5
Thrown From: go-find-references package, v1.2.3
Stack Trace
Uncaught TypeError: Cannot read property 'substring' of undefined
At /Users/jeronimo/.atom/packages/go-find-references/lib/go-find-references-view.coffee:65

TypeError: Cannot read property 'substring' of undefined
    at GoFindReferencesView.module.exports.GoFindReferencesView.trigger (/Users/jeronimo/.atom/packages/go-find-references/lib/go-find-references-view.coffee:65:25)
    at atom-text-editor.<anonymous> (/Users/jeronimo/.atom/packages/go-find-references/lib/go-find-references-view.coffee:43:8)
    at CommandRegistry.module.exports.CommandRegistry.handleCommandEvent (/Users/jeronimo/Downloads/Atom.app/Contents/Resources/app.asar/src/command-registry.js:260:29)
    at /Users/jeronimo/Downloads/Atom.app/Contents/Resources/app.asar/src/command-registry.js:3:61
    at KeymapManager.module.exports.KeymapManager.dispatchCommandEvent (/Users/jeronimo/Downloads/Atom.app/Contents/Resources/app.asar/node_modules/atom-keymap/lib/keymap-manager.js:580:16)
    at KeymapManager.module.exports.KeymapManager.handleKeyboardEvent (/Users/jeronimo/Downloads/Atom.app/Contents/Resources/app.asar/node_modules/atom-keymap/lib/keymap-manager.js:388:22)
    at WindowEventHandler.module.exports.WindowEventHandler.handleDocumentKeyEvent (/Users/jeronimo/Downloads/Atom.app/Contents/Resources/app.asar/src/window-event-handler.js:98:36)
    at HTMLDocument.<anonymous> (/Users/jeronimo/Downloads/Atom.app/Contents/Resources/app.asar/src/window-event-handler.js:3:61)

Commands
     -0:08.3.0 application:add-project-folder (atom-text-editor.editor.is-focused)
     -0:00.7.0 go-find-references:toggle (atom-text-editor.editor.is-focused)

Config
{
  "core": {
    "disabledPackages": [
      "go-debug"
    ],
    "themes": [
      "atom-dark-ui",
      "atom-dark-syntax"
    ]
  }
}

Installed Packages
# User
atom-typescript, v8.3.0
autocomplete-go, v1.1.0
builder-go, v1.0.0
go-config, v1.1.4
go-find-references, v1.2.3
go-get, v1.0.3
go-plus, v4.1.0
gofmt, v1.1.6
gometalinter-linter, v1.1.0
gorename, v1.0.2
linter, v1.11.3
linter-golint, v0.3.0
navigator-godef, v1.0.3
tester-go, v1.0.4

# Dev
No dev packages

The error take places just on an unsaved new file, because filepath = buffer.getPath() is undef.


Empty tuples are formatted with a list inside them

iex> ExFormat.format("{}")
"{[]}\n"

Thanks for the bug report!
Fix: https://github.com/uohzxela/ex_format/commit/fb0bb49f3dff5a92bec7251a89891243dfed1fd4


create namespace when does not exist

Signed-off-by: thalles<EMAIL_ADDRESS>Issue #, if available:
#19
Description of changes:
Now create namespace if does not exist on cloudmap when creating a service
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

Looks good to me, signing off on my end.
Thanks for the improvements and being the project's first public contributor!
I will have to make some modifications when I rebase my latest pull request to modularize the client and modify polling logic, but the implementation is good.


Fixed a misspelling.

Hello, I found a misspelling in isDuplicateName function name. Fixed it.

Codecov Report

Merging #906 into master will decrease coverage by <.01%.
The diff coverage is 2.94%.

@@            Coverage Diff            @@
##           master    #906      +/-   ##
=========================================
- Coverage   55.01%     55%   -0.01%
=========================================
  Files          41      41
  Lines       10057   10057
=========================================
- Hits         5533    5532       -1
- Misses       3499    3500       +1
  Partials     1025    1025

Impacted Files
Coverage Δ

zduplicate.go
4.24% <0%> (ø)
:arrow_up:

duplicate.go
37.5% <50%> (ø)
:arrow_up:

server.go
65.8% <0%> (-0.24%)
:arrow_down:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 8fc2e57...29fdd79. Read the comment docs.


Improved UI + Updated polling times + reduced Contract Reader calls

@amogh-jrules is this the latest to review?

@azf20 Not anymore 😅I'll push a few more commits and then tag you :)


Make nerve service namespace available

This feature is capital when someone wants to enable the services whitelist in NerveHTTPD collector and filter by service name and namespace!

Apart from above 2 very minor problems, it looks good. fix & ship!

:shipit:


TINKERPOP-1033: Store sideEffects as a persisted RDD

https://issues.apache.org/jira/browse/TINKERPOP-1033
This is a massive amount of work. Just making sideEffects be stored as persisted RDDs led to a swath of other updates. Here is the list of things:

It is now possible for Spark users to completely avoid using HDFS -- they simply use PersistedInputRDD and PersistedOutputRDD for everything.
Added a significant amount of testing to ensure that persisted RDDs work as expected in all situations.
InputRDDs now have a readMemoryRDD() method which handles reading sideEffects (i.e. memory).
OutputRDDs now have a writeMemoryRDD() method which handles writing sideEffects (i.e. memory).
There is a Storage interface in gremlin-core which providers can implement to have "file-system semantics" for their data source. HDFS and Spark both implement it. No more Groovy meta-programming for HDFS! Sweeeeet.
With Storage all the file management in both Spark and Giraph is much simpler as the methods in Storage allowed me to gut alot of (error prone) code.
There is a general test suite which makes sure both HDFS and Spark storage behave "the same."
Updated documentation, upgrade docs, and added JavaDoc to Storage.
The docs for BulkLoaderVertexProgram and Spark/Giraph uses a data/ directory. It wasn't consistent with our other examples so I cleaned it up.
Fixed a minor bug in ClusterCountMapReduce.
Cleaned up how HDFS data is streamed -- its pure now, based solely on InputFormat behavior (I learned something new in Hadoop).
There are a few minor "breaking changes" around hdfs.methods(). They are "ok" as HDFS interaction prior to this moment  has always been manual via the Gremlin Console.

I updated the "update" docs:
http://tinkerpop.apache.org/docs/3.1.1-SNAPSHOT/upgrade/#_storage_i_o
I updated the "reference" docs:
http://tinkerpop.apache.org/docs/3.1.1-SNAPSHOT/reference/#_storage_systems
You can see the JavaDoc for the new Storage interface:
http://tinkerpop.apache.org/javadocs/3.1.1-SNAPSHOT/core/org/apache/tinkerpop/gremlin/structure/io/Storage.html
I ran integration tests and built and deployed docs successfully.
VOTE +1.

Integration test BUILD SUCCESSFUL.
[INFO] Apache TinkerPop .................................. SUCCESS [4.800s]
[INFO] Apache TinkerPop :: Gremlin Shaded ................ SUCCESS [2.300s]
[INFO] Apache TinkerPop :: Gremlin Core .................. SUCCESS [34.224s]
[INFO] Apache TinkerPop :: Gremlin Test .................. SUCCESS [11.772s]
[INFO] Apache TinkerPop :: Gremlin Groovy ................ SUCCESS [32.672s]
[INFO] Apache TinkerPop :: Gremlin Groovy Test ........... SUCCESS [6.828s]
[INFO] Apache TinkerPop :: TinkerGraph Gremlin ........... SUCCESS [3:22.133s]
[INFO] Apache TinkerPop :: Hadoop Gremlin ................ SUCCESS [5:02.151s]
[INFO] Apache TinkerPop :: Spark Gremlin ................. SUCCESS [4:03.723s]
[INFO] Apache TinkerPop :: Giraph Gremlin ................ SUCCESS [2:01:35.469s]
[INFO] Apache TinkerPop :: Neo4j Gremlin ................. SUCCESS [18:06.653s]
[INFO] Apache TinkerPop :: Gremlin Driver ................ SUCCESS [10.940s]
[INFO] Apache TinkerPop :: Gremlin Server ................ SUCCESS [11:13.160s]
[INFO] Apache TinkerPop :: Gremlin Console ............... SUCCESS [1:10.880s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 2:46:18.192s
[INFO] Finished at: Wed Jan 06 19:01:26 MST 2016
[INFO] Final Memory: 103M/708M

mvn clean install: passed
integration tests: passed
bin/process-docs.sh: passed

VOTE: +1

This PR drops a few "internal" classes - folks shouldn't have been using those directly, but would it have been better to deprecate those as opposed to just removing completely?
Seems like deprecation would have worked for:

[HadoopLoader](https://github.com/apache/incubator-tinkerpop/pull/192/files#diff-* 55e3610726b342e666b34223b8270526)
HDFSTools
SparkLoader

even though these are pretty low-level classes, it would be nice to stick to "not breaking (even if they are doing stuff they are not supposed to)" plan imo.

Here is the thing. SparkLoader was introduced in 3.1.1-SNAPSHOT :D so it okay to drop. HadoopLoader is all meta-programming Groovy stuff to get ls(), rm(), etc. to work in Gremlin Console. We can keep the the class, but we can't have it loaded else it will interfere with the new FileSystemStorage. However, I say we just drop it. Its so low level and all meta-programmy that if someone is using it, they are retarded.
Finally, HDFSTools. Again, low level.... I can bring that class back, but people really shouldn't be using it. This is like an internal utility and so specific to TinkerPop filesystem stuff. ??

Yeah - they shouldn't be using them, but we know how that goes....make a class public and someone is gonna use it to their detriment or otherwise.  anyway, i'd say just do one of the following then:

bring them back and deprecate
add something to upgrade docs to explain their removal - you already have the section - perhaps just a WARNING that explicitly mentions the classes.

VOTE: +1