My process being killed the moment it start training

I am trying to train image detector using my own dataset but it fails. I've made few changes in configuration file apart from number of classes and paths. Here are they
train_input_reader: {
  tf_record_input_reader {
    input_path: "data/data_train.record"
  }
  label_map_path: "data/rdata_train.pbtxt"
  queue_capacity: 2
  min_after_dequeue: 1
}

train_config: {
  batch_size: 1
  optimizer {
    rms_prop_optimizer: {
      learning_rate: {
        exponential_decay_learning_rate {
          initial_learning_rate: 0.1
          decay_steps: 800720
          decay_factor: 0.95
        }
      }
      momentum_optimizer_value: 0.9
      decay: 0.9
      epsilon: 0.000001
    }
  }
  fine_tune_checkpoint: "ssd_mobilenet_v1_coco_2017_11_17/model.ckpt"
  from_detection_checkpoint: true
  # Note: The below line limits the training process to 200K steps, which we
  # empirically found to be sufficient enough to train the pets dataset. This
  # effectively bypasses the learning rate schedule (the learning rate will
  # never decay). Remove the below line to train indefinitely.
  num_steps: 200000
  batch_queue_capacity: 5
  num_batch_queue_threads: 8
  prefetch_queue_capacity: 5
}

But it fails to start training. I checked issues on tensorflow research repository but no specific answer I was able to find.
/home/ubuntu/.conda/envs/tf/lib/python3.6/site-packages/h5py/__init__.py:36: FutureWarning: Conversion of the second argument of issubdtype from `float` to `np.floating` is deprecated. In future, it will be treated as `np.float64 == np.dtype(float).type`.
  from ._conv import register_converters as _register_converters
WARNING:tensorflow:From /dev/models/research/object_detection/trainer.py:228: create_global_step (from tensorflow.contrib.framework.python.ops.variables) is deprecated and will be removed in a future version.
Instructions for updating:
Please switch to tf.train.create_global_step
INFO:tensorflow:depth of additional conv before box predictor: 0
INFO:tensorflow:depth of additional conv before box predictor: 0
INFO:tensorflow:depth of additional conv before box predictor: 0
INFO:tensorflow:depth of additional conv before box predictor: 0
INFO:tensorflow:depth of additional conv before box predictor: 0
INFO:tensorflow:depth of additional conv before box predictor: 0
INFO:tensorflow:Summary name /clone_loss is illegal; using clone_loss instead.
WARNING:tensorflow:From /home/ubuntu/.conda/envs/tf/lib/python3.6/site-packages/tensorflow/contrib/slim/python/slim/learning.py:736: Supervisor.__init__ (from tensorflow.python.training.supervisor) is deprecated and will be removed in a future version.
Instructions for updating:
Please switch to tf.train.MonitoredTrainingSession
2018-03-22 14:14:27.789420: I tensorflow/core/platform/cpu_feature_guard.cc:140] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
2018-03-22 14:14:27.925567: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:898] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2018-03-22 14:14:27.925895: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1212] Found device 0 with properties:
name: Tesla K80 major: 3 minor: 7 memoryClockRate(GHz): 0.8235
pciBusID: 0000:00:1e.0
totalMemory: 11.17GiB freeMemory: 11.10GiB
2018-03-22 14:14:27.925920: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1312] Adding visible gpu devices: 0
2018-03-22 14:14:29.848609: I tensorflow/core/common_runtime/gpu/gpu_device.cc:993] Creating TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 10764 MB memory) -> physical GPU (device: 0, name: Tesla K80, pci bus id: 0000:00:1e.0, compute capability: 3.7)
INFO:tensorflow:Restoring parameters from ckpts/model.ckpt-0
INFO:tensorflow:Running local_init_op.
INFO:tensorflow:Done running local_init_op.
INFO:tensorflow:Starting Session.
INFO:tensorflow:Saving checkpoint to path ckpts/model.ckpt
INFO:tensorflow:Starting Queues.
INFO:tensorflow:global_step/sec: 0
INFO:tensorflow:Recording summary at step 0.
Killed

Please help
Many thanks!!!

Can you please post on stack overflow?
Sent from my iPhone

On Mar 22, 2018, at 10:13 AM, Aditya Thakkar<EMAIL_ADDRESS>wrote:
I am trying to train image detector using my own dataset but it fails. I've made few changes in configuration file apart from number of classes and paths. Here are they
train_input_reader: {
tf_record_input_reader {
input_path: "data/data_train.record"
}
label_map_path: "data/rdata_train.pbtxt"
queue_capacity: 2
min_after_dequeue: 1
}
train_config: {
batch_size: 1
optimizer {
rms_prop_optimizer: {
learning_rate: {
exponential_decay_learning_rate {
initial_learning_rate: 0.1
decay_steps: 800720
decay_factor: 0.95
}
}
momentum_optimizer_value: 0.9
decay: 0.9
epsilon: 0.000001
}
}
fine_tune_checkpoint: "ssd_mobilenet_v1_coco_2017_11_17/model.ckpt"
from_detection_checkpoint: true
Note: The below line limits the training process to 200K steps, which we
empirically found to be sufficient enough to train the pets dataset. This
effectively bypasses the learning rate schedule (the learning rate will
never decay). Remove the below line to train indefinitely.
num_steps: 200000
batch_queue_capacity: 5
num_batch_queue_threads: 8
prefetch_queue_capacity: 5
}
But it fails to start training. I checked issues on tensorflow research repository but no specific answer I was able to find.
Please help
Many thanks!!!
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub, or mute the thread.

Hello. It happens because of your RAM memory size.

I am trying to train image detector using my own dataset but it fails. I've made few changes in configuration file apart from number of classes and paths. Here are they
train_input_reader: {
  tf_record_input_reader {
    input_path: "data/data_train.record"
  }
  label_map_path: "data/rdata_train.pbtxt"
  queue_capacity: 2
  min_after_dequeue: 1
}

train_config: {
  batch_size: 1
  optimizer {
    rms_prop_optimizer: {
      learning_rate: {
        exponential_decay_learning_rate {
          initial_learning_rate: 0.1
          decay_steps: 800720
          decay_factor: 0.95
        }
      }
      momentum_optimizer_value: 0.9
      decay: 0.9
      epsilon: 0.000001
    }
  }
  fine_tune_checkpoint: "ssd_mobilenet_v1_coco_2017_11_17/model.ckpt"
  from_detection_checkpoint: true
  # Note: The below line limits the training process to 200K steps, which we
  # empirically found to be sufficient enough to train the pets dataset. This
  # effectively bypasses the learning rate schedule (the learning rate will
  # never decay). Remove the below line to train indefinitely.
  num_steps: 200000
  batch_queue_capacity: 5
  num_batch_queue_threads: 8
  prefetch_queue_capacity: 5
}

But it fails to start training. I checked issues on tensorflow research repository but no specific answer I was able to find.
/home/ubuntu/.conda/envs/tf/lib/python3.6/site-packages/h5py/__init__.py:36: FutureWarning: Conversion of the second argument of issubdtype from `float` to `np.floating` is deprecated. In future, it will be treated as `np.float64 == np.dtype(float).type`.
  from ._conv import register_converters as _register_converters
WARNING:tensorflow:From /dev/models/research/object_detection/trainer.py:228: create_global_step (from tensorflow.contrib.framework.python.ops.variables) is deprecated and will be removed in a future version.
Instructions for updating:
Please switch to tf.train.create_global_step
INFO:tensorflow:depth of additional conv before box predictor: 0
INFO:tensorflow:depth of additional conv before box predictor: 0
INFO:tensorflow:depth of additional conv before box predictor: 0
INFO:tensorflow:depth of additional conv before box predictor: 0
INFO:tensorflow:depth of additional conv before box predictor: 0
INFO:tensorflow:depth of additional conv before box predictor: 0
INFO:tensorflow:Summary name /clone_loss is illegal; using clone_loss instead.
WARNING:tensorflow:From /home/ubuntu/.conda/envs/tf/lib/python3.6/site-packages/tensorflow/contrib/slim/python/slim/learning.py:736: Supervisor.__init__ (from tensorflow.python.training.supervisor) is deprecated and will be removed in a future version.
Instructions for updating:
Please switch to tf.train.MonitoredTrainingSession
2018-03-22 14:14:27.789420: I tensorflow/core/platform/cpu_feature_guard.cc:140] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
2018-03-22 14:14:27.925567: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:898] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2018-03-22 14:14:27.925895: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1212] Found device 0 with properties:
name: Tesla K80 major: 3 minor: 7 memoryClockRate(GHz): 0.8235
pciBusID: 0000:00:1e.0
totalMemory: 11.17GiB freeMemory: 11.10GiB
2018-03-22 14:14:27.925920: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1312] Adding visible gpu devices: 0
2018-03-22 14:14:29.848609: I tensorflow/core/common_runtime/gpu/gpu_device.cc:993] Creating TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 10764 MB memory) -> physical GPU (device: 0, name: Tesla K80, pci bus id: 0000:00:1e.0, compute capability: 3.7)
INFO:tensorflow:Restoring parameters from ckpts/model.ckpt-0
INFO:tensorflow:Running local_init_op.
INFO:tensorflow:Done running local_init_op.
INFO:tensorflow:Starting Session.
INFO:tensorflow:Saving checkpoint to path ckpts/model.ckpt
INFO:tensorflow:Starting Queues.
INFO:tensorflow:global_step/sec: 0
INFO:tensorflow:Recording summary at step 0.
Killed

Please help
Many thanks!!!

Yes I am also facing the same problem and my training my model over google colab. Is there any solution ?

I am trying to train image detector using my own dataset but it fails. I've made few changes in configuration file apart from number of classes and paths. Here are they
train_input_reader: {
  tf_record_input_reader {
    input_path: "data/data_train.record"
  }
  label_map_path: "data/rdata_train.pbtxt"
  queue_capacity: 2
  min_after_dequeue: 1
}

train_config: {
  batch_size: 1
  optimizer {
    rms_prop_optimizer: {
      learning_rate: {
        exponential_decay_learning_rate {
          initial_learning_rate: 0.1
          decay_steps: 800720
          decay_factor: 0.95
        }
      }
      momentum_optimizer_value: 0.9
      decay: 0.9
      epsilon: 0.000001
    }
  }
  fine_tune_checkpoint: "ssd_mobilenet_v1_coco_2017_11_17/model.ckpt"
  from_detection_checkpoint: true
  # Note: The below line limits the training process to 200K steps, which we
  # empirically found to be sufficient enough to train the pets dataset. This
  # effectively bypasses the learning rate schedule (the learning rate will
  # never decay). Remove the below line to train indefinitely.
  num_steps: 200000
  batch_queue_capacity: 5
  num_batch_queue_threads: 8
  prefetch_queue_capacity: 5
}

But it fails to start training. I checked issues on tensorflow research repository but no specific answer I was able to find.
/home/ubuntu/.conda/envs/tf/lib/python3.6/site-packages/h5py/__init__.py:36: FutureWarning: Conversion of the second argument of issubdtype from `float` to `np.floating` is deprecated. In future, it will be treated as `np.float64 == np.dtype(float).type`.
  from ._conv import register_converters as _register_converters
WARNING:tensorflow:From /dev/models/research/object_detection/trainer.py:228: create_global_step (from tensorflow.contrib.framework.python.ops.variables) is deprecated and will be removed in a future version.
Instructions for updating:
Please switch to tf.train.create_global_step
INFO:tensorflow:depth of additional conv before box predictor: 0
INFO:tensorflow:depth of additional conv before box predictor: 0
INFO:tensorflow:depth of additional conv before box predictor: 0
INFO:tensorflow:depth of additional conv before box predictor: 0
INFO:tensorflow:depth of additional conv before box predictor: 0
INFO:tensorflow:depth of additional conv before box predictor: 0
INFO:tensorflow:Summary name /clone_loss is illegal; using clone_loss instead.
WARNING:tensorflow:From /home/ubuntu/.conda/envs/tf/lib/python3.6/site-packages/tensorflow/contrib/slim/python/slim/learning.py:736: Supervisor.__init__ (from tensorflow.python.training.supervisor) is deprecated and will be removed in a future version.
Instructions for updating:
Please switch to tf.train.MonitoredTrainingSession
2018-03-22 14:14:27.789420: I tensorflow/core/platform/cpu_feature_guard.cc:140] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
2018-03-22 14:14:27.925567: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:898] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2018-03-22 14:14:27.925895: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1212] Found device 0 with properties:
name: Tesla K80 major: 3 minor: 7 memoryClockRate(GHz): 0.8235
pciBusID: 0000:00:1e.0
totalMemory: 11.17GiB freeMemory: 11.10GiB
2018-03-22 14:14:27.925920: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1312] Adding visible gpu devices: 0
2018-03-22 14:14:29.848609: I tensorflow/core/common_runtime/gpu/gpu_device.cc:993] Creating TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 10764 MB memory) -> physical GPU (device: 0, name: Tesla K80, pci bus id: 0000:00:1e.0, compute capability: 3.7)
INFO:tensorflow:Restoring parameters from ckpts/model.ckpt-0
INFO:tensorflow:Running local_init_op.
INFO:tensorflow:Done running local_init_op.
INFO:tensorflow:Starting Session.
INFO:tensorflow:Saving checkpoint to path ckpts/model.ckpt
INFO:tensorflow:Starting Queues.
INFO:tensorflow:global_step/sec: 0
INFO:tensorflow:Recording summary at step 0.
Killed

Please help
Many thanks!!!

Yes. I am also facing the same problem and I am training my model over google colab. Is there any solution ?

My issue is resolved as I resized the training images.


ARROW-14426: [C++] Add a minimum_row_group_size to dataset writing

This adds min_rows_per_group and max_rows_per_group to arrow::dataset::FileSystemDatasetWriteOptions which give the user control over the size of row groups that are created by the dataset writer.

Is there anything else that needs to be done on this?

Some of these failures seem like legitimate deadlocks unfortunately.  I'll try and reproduce.

Rebased.  Found and fixed a deadlock / race condition in the serialized async task group.  The only build error now appears to be an unrelated plasma test.
@lidavidm I assume you're still good with this getting merged in?  I don't know if you wanted to take a glance at the commit I added or not.

LGTM, thanks. I'll merge now.


Implement tests for autocmd commands

ALE relies on autocmd commands for text being changed, buffers being opened, and files being saved, etc. It would be nice to have some Vader tests which check that when these events occur, that ALE behaves as expected. I have had some trouble getting a Vader test for this to actually work. If anyone is able to get this working, that would be very helpful.

Some autocommands have to be triggered manually, others (e.g. InsertLeave) work.


Compliance scanning for cloud providers

Would be awesome if we could use hubblestack to scan for compliance against cloud provider benchmarks, such as this one, https://aws.amazon.com/blogs/security/tag/cis-aws-foundations-benchmark/

This would be great! Thanks for the request!

This issue was moved to hubblestack/hubble-salt#45


Development: Track Number of Server Starts in Tests

Checklist
General

[x] Language: I followed the guidelines for inclusive, diversity-sensitive, and appreciative language.
[x] I chose a title conforming to the naming conventions for pull requests.

Motivation and Context

Restarting the server when the tests are run is very time expensive and we want to ensure it does not happen too often. If it does, the newly added task Number of Server Starts fails.
Description

This PR adds a task tracking how many times the server has been started in tests. The task fails if the server has been started more than 4 times. Additionally, the number of starts is printed to the console.
Review Progress

Code Review

[ ] Code Review 1
[ ] Code Review 2

Test Coverage

Codecov Report

Merging #6738 (29a352e) into develop (4e4d92a) will decrease coverage by 4.59%.
The diff coverage is n/a.

:exclamation: Your organization is not using the GitHub App Integration. As a result you may experience degraded service beginning May 15th. Please install the Github App Integration for your organization. Read more.
Additional details and impacted files
@@              Coverage Diff              @@
##             develop    #6738      +/-   ##
=============================================
- Coverage      84.95%   80.37%   -4.59%
- Complexity     13911    13912       +1
=============================================
  Files           1156     2448    +1292
  Lines          44886    93346   +48460
  Branches        4623    13105    +8482
=============================================
+ Hits           38135    75028   +36893
- Misses          4226    10088    +5862
- Partials        2525     8230    +5705

see 1299 files with indirect coverage changes

Flag
Coverage Δ

client
76.12% <ø> (?)

server
84.96% <ø> (+<0.01%)
:arrow_up:

Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 4e4d92a...29a352e. Read the comment docs.


Initialize nextInChain to nil in extensible getters.

Fixes a crash on macOS when calling (some?) extensible getters (e.g. Adapter.properties). It seems to work fine on Windows without this patch, maybe Swift runtime does something differently on other platforms, not sure.
Extensible getters create the associated C struct (e.g. WGPUAdapterProperties), but it does not reset their nextInChain property to nil, so Dawn crashes thinking that it's pointing to some real thing.
According to Xcode docs, Swift will zero all values after allocating an imported C struct, but apparently it doesn't always do that.

That's very strange. Anyway, it's probably better to do it explicitly. Thanks!


I don't like the button in the left side bar

Hi @tagg-lee, in version 0.6.5 you're able to hide the button ❤

Thank you for the update!


Find false errors in docstring

def unicode_foo():
    """
    Returns "foo" as a str in python3 and as a
    unicode in python2
    """
    return six.text_type("foo")

After applying six --write unicode on the file, my script looks like
def unicode_foo():
    """
    Returns "foo" as a str in python3 and as a
    six.text_type in python2
    """
    return six.text_type("foo")

I'm using sixer==1.6.1

sixer uses regex and isn't safe. As you noticed, it also modify docstrings, especially for the "unicode" operation. You can exclude this operation, or revert manually changes. You may try other tools: https://github.com/vstinner/sixer/#resources-to-port-code-to-python-3
I don't see how to fix the unicode operation since it only matchs the "unicode" string...


Changes CertificateRequest conditions to only Pending/Failed/Issued

This condenses the condition status reasons for the CertificateRequest to Pending, Failed and Issued
This simplifies how a controller will handle states of this resource.
/assign @munnerz
NONE

Addressed the comments :)

/lgtm
/approve

/retest


attribute "is_searched" not found because Attention not converted to SparseAttention

As shown in the figure, there are errors reported about the attribute "is_searched" not found. After tracing back this error, I found the problem is because in the class ModuleInjection, it always go to the if condition and return the model without converting the attention layer into SparseAttention. "is_searched" is an attribute only exists in SparseAttention. (For now to solve this problem, I commented out the if branch, but this will bring up new errors.) Is there any solution for this error? Thanks!

@xjaweiofj hi, same error. Do you know how to fix it?

+1


Create Student Record Management System using C/C++/python/java

Student Record Management System Student Record Management System  is a program that can organize and maintain the list of all students. The sole purpose of the system is to have a reliable and convenient of maintaining the record of all students inside the campus. The project can be access freely without providing a login information. The user do many things, he/she can register new student, update information, check all the student details, and also can delete existing student record

Hey!, I can work on this issue. Please assign this issue to me.

I have already completed it in java. You can try in another language.


Log to console mode

@johncburns1: Can you do me a favor and add a somewhat hardcoded mode to the cadence-proxy that will print the log events out to standard output in addition to transmitting them to the .NET client via LogRequest messages?

[ ] The idea is that this would be an internal global boolean constant or variable that normally be FALSE but it would be nice to be able to temporarily be able to enable this when debugging.

[ ] Could you also log the client ID for all messages sent or received.  I'm currently debugging some multi-connection issues and this info would come in handy.

Would you like this to be able to be toggled or just hardcode it to be enabled?

Yeah, it would make sense to hardcode this mode when cadence-proxy is run as its own process but never when it's deployed as a shared library.  That would be the best of all worlds.

I am about to push the changes.  I created a boolean var in main.go called verboseLogging.  When set to true, this sets the global var VerboseLogging to true.  Works exactly the same as DebugPrelaunched.  I altered the custom logger setup to take both debugPrelaunched and verboseLogging as parameters to create the logger core with.
Rules for the logger are this:
func (c Core) Write(entry zapcore.Entry, fields []zapcore.Field) error {
    if !debugPrelaunched {
        return transmitToDotnetClient()
    }
    var err error
    err = logToStandardOutput()
    if err != nil {
        return err
    }
    if verboseLogging {
        err = transmitToDotnetClient()
        if err != nil {
            return err
        }
    }
    return nil
}

I made the changes.  If the cadence-proxy is run with the debug flag (passed via command line args), then output will be logged to both stdout and transmitted to the .NET cadence client.  This works for both DebugPrelaunch and normal debugging.

OK.  We'll call this done now and close this issue.


Track *.png and *.jpg using Git LFS

What changed?

git lfs track "*.png"
git lfs track "*.jpg"
git add **/*.png

Rationale

Configure Git LFS, so binaries don't blew up the repo size.

How was it tested?
If it was manually verified, list the instructions for your reviewers to follow.

[ ] Unit Tests
[ ] Functional Tests
[x] Manually Verified

Do we need instructions somewhere? Like here: https://docs.iambic.org/how_to_guides/contributing , or in our README.md (if it makes sense there)?

Do we need instructions somewhere? Like here: https://docs.iambic.org/how_to_guides/contributing , or in our README.md (if it makes sense there)?

added https://github.com/noqdev/iambic/pull/551/commits/c6a2f6a15b3b6430d61005d2b82f3ca6fea6c227


How do I run this with a debugger?

  I can't seem to get create-data-mod running with the gdb debugger.  Are there steps you all have taken to get that working?  I tried building with debug flags:
cmake -DSTATIC_COMPILE=TRUE -DCMAKE_BUILD_TYPE=Debug ..
cmake --build .

And I also tried adding set(CMAKE_BUILD_TYPE Debug) to CMakeLists.txt
Is there something I'm missing?  Shouldn't that add the -g flag to the g++ compiler under the hood so we can run gdb with symbol tables?
But when I do gdb create-data-mod I get: (No debugging symbols found in create-data-mod)
Once again I apologize if this question seems stupid, I'm pretty rusty with my c++ and makefiles

I figured it out, I strangely had to set custom cmake flags to FORCE the compiler to be g++:
cmake -DCMAKE_C_COMPILER:FILEPATH=/usr/bin/gcc -DCMAKE_CXX_COMPILER:STRING=/usr/bin/g++ -DCMAKE_BUILD_TYPE=Debug ..

Also, just fyi the main reason I was trying to get the debugger working was to better understand the logic happening under the hood for random cost mod.  My friends' and I are big fans and we had some ideas to make it better, mainly setting lower bounds for unit costs and making sure castles aren't too cheap, because in these instances the games quickly devolve into two metas: castle spam, or unit spam while disregarding economy.  Of course, not trying to step on yall's toes so if these aren't adjustments you want in the official random cost mod I can publish a separate mod for just me and my friends.

I don't want to step on this merge request's toes: #11 which seems to already have changes to the readme? @simonsan should I work off of that pull request to update the docs?

Don't worry, I can rebase it and move stuff around once your PR is merged. ;) Thanks for asking though!

Also, just fyi the main reason I was trying to get the debugger working was to better understand the logic happening under the hood for random cost mod. My friends' and I are big fans and we had some ideas to make it better, mainly setting lower bounds for unit costs and making sure castles aren't too cheap, because in these instances the games quickly devolve into two metas: castle spam, or unit spam while disregarding economy. Of course, not trying to step on yall's toes so if these aren't adjustments you want in the official random cost mod I can publish a separate mod for just me and my friends.

I would be surprised if the debugger tells you more about how it works than reading the source code, but you do you :sweat_smile:
There is only one safeguard implemented currently, and I am under the impression that the whacky generations that this permits are part of the appeal of this mod.
If you want to look into adding further safeguards, I think they would ideally be implemented as toggleable via command line parameters, so we can continue to ship an "official" version while others can activate additional safeguards as they please for their versions.

@HSZemi thanks for the feedback!  I have a couple of questions and notes.
My friends and I discovered the random mod a few weeks ago and have been avid fans.  For the first few games, it was fun every time, but after a couple days we quickly noticed that most of random mod games had devolved into one or both of these two fairly predictable metas:

Castles are cheap.  Castles are normally 650 stone, so in the random mod if they are significantly cheaper (they almost always are) the game devolves to castle spam around your base with a race to a wonder.

Discover spam unit with a very low cost as quickly as possible.  In many scenarios, the random cost mod devolves into an aging up race to see who can discover a cheap spammy unit the fastest.  If someone discovers a spam unit just slightly before someone else, the game is over, as they will be overrun others without time to respond.  Furthermore, the spam unit negates the need to maintain an economy, and so a major element of gameplay.

I believe random mod has a replay value problem.  Those above metas are hilarious and a joy the first few times.  After that, they become tedious and repetitive.  I think a random mod, ultimately, should give a variety of random experiences and fresh strategies to players so they always need to think on their feet, but if the random mod usually falls into two metas this is no longer the case.  I think one of the beauties of the random mod itself is that it DESTROYS the traditional AOE2 metas.  Every day, there's a new meta.  It feels fresh and new and it forces you to think out of the box, after you've been playing AOE for so long and dust has settled on your brain and you can basically play the game without thinking.
But I agree that the made random mod is so fun the first few times that it should probably remain as it is.  I propose a new mod, something like "Random Mod Balanced" targeted with some sane defaults, like preventing castle spam and units costing 1 wood.  Is this something you would considering publishing through Siege Engineers if I make the code mods for it?  A lot of my friends have stopped playing Random Mod because of the metas and they said they'd be interested to come back if we there was a more balanced version that prevented those two metas I mentioned.
If not, do you mind giving me some information on how to set up a build system?  Is it a lot of work to set up?  Do ya'll have a server somewhere that publishes the new random mod every day?  Is that script in the repo here?  I don't see any devops scripts.  And how do you detect when new AOE patches go out, and thus when a new fresh dat is available for moding?  Does AOE provide an api or something for getting the fresh dat file?
Thanks for your time!


First stab at Windows support

Checklist

[x] Used a personal fork of the feedstock to propose changes
[x] Bumped the build number (if the version is unchanged)
[ ] Reset the build number to 0 (if the version changed)
[ ] Re-rendered with the latest conda-smithy (Use the phrase @conda-forge-admin, please rerender in a comment in this PR for automated rerendering)
[ ] Ensured the license file is being packaged.

Try enabling Windows support using MS-MPI. Tagging @msmdev just to be in the loop.
Related CellRank issue: https://github.com/theislab/cellrank/issues/776

Hi! This is the friendly automated conda-forge-linting service.
I just wanted to let you know that I linted all conda-recipes in your PR (recipe) and found it was in an excellent condition.

@conda-forge-admin, please rerender

Looking at [https://petsc.org/release/install/windows/](PETSc installation instructions), it seems manual Windows install is cumbersome/not recommended and on conda-forge, petsc{4py} is also not set up for Windows (+ no MS-MPI support).
Until something changes, on that side, I am closing this.


Re-enable stats-based replica rebalancing

This means turning on both load-based replica rebalancing and size-based replica rebalancing. So in addition to some how-to docs, we'll need to update our architecture docs.
Background: https://airtable.com/tblD3oZPLJgGhCmch/viwINAgXA5on9pr2T/recWJ095g7CriLJ88
PM: @awoods187
Eng Partner: @a-robinson

PR: https://github.com/cockroachdb/cockroach/pull/28340
From release notes:

Range leases are now automatically rebalanced throughout the cluster to even out the amount of QPS being handled by each node. [#28340][#28340] {% comment %}doc{% endcomment %}

@a-robinson, is this work now done and ready to be documented?

Not quite, I'd suggest waiting until next week.

From forum:

The decisions are based on the number of queries processed by a node made over the last half hour or so using an exponentially weighted moving average to more heavily weight recent load. They only happen if load on the cluster is above a certain threshold, the difference in load between nodes is substantial (greater than 25% difference by default), and there are leases that could be transferred that will meaningfully even out the load in the cluster.
Only leases are transferred based on load as of today. Very little I/O is involved in lease transfers, since the data itself doesn’t have to be moved from the nodes it’s currently on in order to change the leaseholder.


Which pip package should I install?

python-zenity or zenipy?

zenipy, python-zenity is the old one


Adding a flag to only output the token

Why:

This enables you to use the authenticator binary to generate
short-lived access to tools like the Kubernetes Dashboard using the
token field

This change addresses the need by:

closes #85

Usecase:
aws-iam-authenticator -i <cluster-name> --only-token

Signed-off-by: Christopher Hein<EMAIL_ADDRESS>

Nice.

Just wanted to point out that the actual flag is --token-only.  Also, it would be awesome if this (and maybe k8s dashboard usage with this project) were mentioned in the readme.


chtbl.com

I understand why you'd want to add it to the list (it's a tracking domain, and most people that use lists like these don't want to be tracked), but in this case, I don't think it should be blocked.
My reasoning for this: Pocket Casts (or really, any podcast app) is unable to play certain podcasts I like to listen to that use Chartable for their analytics.
Edit: I probably should have mentioned the following:

I'm using a Pi-Hole, and whitelisting the domain fixes the issue.
Looks like it was added in 82c18786d.

Thanks @SheepKid12.  This is upstream.
Ping @jawz101 care to comment?  Whatever you decide.

Can confirm on Overcast app on iOS as well. Episodes failed to download until I whitelisted chtbl.com.

removed.  thanks


Metadata: Adds initial support for entity type inheritance.

Also some work on setting up fixtures for inheritance FTs.
NB: Metadata support is not complete here. This should be enough to get started on feature work though.

Shouldn't Indexes, FKs, etc also be inherited or is that will just come later?

@AndriySvyryd That will come later.

:shipit:


Tighten budget source range

Currently foaf:Agent, could/should be some AU Org unit.

Budget might be sourced from many places, including philanthropy, IMF, advertising...

Fair enough


Include hpp and c files in CMakeLists.txt

Which branch your pull-request should merge into?

dev: Current 2.x BugFixs or Features

Describe your changes
Same title
Issue ticket number and link
Checklist before requesting a review
For each PR

[ ] Add Copyright if it missed:
- "Copyright (c) 2019-present Axmol Engine contributors (see AUTHORS.md)."

[ ] I have performed a self-review of my code.
Optional:

[ ] I have checked readme and add important infos to this PR.
[ ] I have added/adapted some tests too.

For core/new feature PR

[ ] I have checked readme and add important infos to this PR.
[ ] I have added thorough tests.

Can you add on Lua template too?

yes, i will


chore: remove deno relay code

Follow up to https://github.com/supabase/cli/pull/1171

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

15 of 25 (60.0%)  changed or added relevant lines in 5 files are covered.
10 unchanged lines in 2 files lost coverage.
Overall coverage increased (+0.1%) to 61.989%

Changes Missing Coverage
Covered Lines
Changed/Added Lines
%

cmd/functions.go
0
1
0.0%

internal/start/start.go
9
18
50.0%

Files with Coverage Reduction
New Missed Lines
%

internal/utils/colors.go
3
50.0%

internal/gen/keys/keys.go
7
7.94%

Totals

Change from base Build<PHONE_NUMBER>:
0.1%

Covered Lines:
4669

Relevant Lines:
7532

💛  - Coveralls


Hitting Configure after file selected does nothing

A bug related to browsers.
Firefox supports "String.concat" function while Chrome supports only "any string".concat()
Resolved by the commit b31d3c4384446a9b74e50fdf79b1fd48729f5d20


Ability to pass the run ID to the training script

First of all, my team is very grateful for your work on this project; thank you.
My question: Is it possible to pass the run ID to the training script so that generated files can be saved into a directory with the same ID? This would make it much easier for us to be able to go from comparing models in guild to pulling the weights and putting a model into our testing environment.
Currently, we are logging the outputs then after comparing the runs we have to go through the logs and find where the model was saved. I see there is a section for files that have been generated by the run, but for MXNet this doesn't seem to ever find anything.
Alternatively, if there is already a better solution I would love to know how to implement it.
Thanks,
Sam

Run ID is always available as the environment variable RUN_ID. You can get the run directory from the RUN_DIR environment variable.
E.g. as sample Guild file:
test:
  exec: bash -c 'echo $RUN_ID - $RUN_DIR'

That said, to save files to the run directory, simply create them with a relative path. Guild changes the working directory to the run directory for your script. This is a major design consideration that lets Guild associate generated files with each unique run directory.
For example, to write a file hello.txt to the run directory:
with open("hello.txt", "w") as f:
    f.write("hi!")

You can view the files associated with a run using the ls command:
$ guild ls  # defaults to showing files for the latest run

It sounds like you're trying to do something using MXNet that looks for files somewhere. This could be a major point of confusion if you or MXNet is assuming that files are generated in a specific location and Guild, by changing the current directory, is misdirecting those files.
If you can elaborate on what sort of errors (specific messages are helpful if you're able to provide them) you're seeing, we can try to resolve the issue.

Just being able to get RUN_ID using os.getenv('RUN_ID') solves my problem, thanks!
I didn't know about these environmental variables, is there a list of these?

Unfortunately I'm seeing no docs covering this topic. I'll make sure those land in the upcoming release. Thanks for pointing that out!
Though just to nag a little, it's uncommon for a script to know or use the run ID. The goal is to support scripts that are independent of Guild - so that you and others can run them without Guild for dev purposes or just to not have to use a tool to do something simple like run a Python script :)
If your script is spending cycles trying to locate generated files using Guild specific environment variables, consider using relative paths - or use explicit paths specified by a command line arg.
For example, here's a training script that uses argparse to define a CLI, which includes the path where files should be generated:
import argparse
import my_model

p = argparse.add_argument(
    "--output-dir",
    default="output",
    help="location of generated files")
args = p.parse_args()

my_model.train(args.output_dir)

Note there's no Guild-specific code in there! When you run the script, output lands in the output subdirectory. In the next step we configure the Guild op to use . (current directory).
Here's the Guild file:
train:
  main: train --output-dir .

When Guild runs a script, it changes the current directory to the unique run directory, so in this case . is the run directory. The script relies on args.output_dir so there's no need to look for RUN_ID or RUN_DIR.
Of course if you're doing something different, ignore this! I just don't want you do have to do a bunch of work in your code if you don't need to :)

When we run experiments we put each generated model into separate directories each named with a random hash (mainly because we generate a lot of outputs). So this just enables us to use the guild id as the hash instead of having to cross-reference between two hashes.
Thanks for all your help, looking forward to seeing the global variable doc in an upcoming release.


SQS Queue docs look wrong

What happened?
https://www.pulumi.com/registry/packages/aws/api-docs/sqs/queue/
Some of the examples for all languages are listed underneath each other, without a language selection menu.
Example
.
Output of pulumi about
.
Additional context
No response
Contributing
Vote on this issue by adding a 👍 reaction.
To contribute a fix for this issue, leave a comment (and link to your pull request, if you've opened one already).

This is a duplicate of: https://github.com/pulumi/pulumi-terraform-bridge/issues/926

This has been fixed with https://github.com/pulumi/pulumi-terraform-bridge/issues/926.


Add metrics relation to tensorboard-controller

Add metrics-endpoint relation to tensorboard-controller and simple alert rules for service being down.
How to check metrics
$ cd charms/tensorboard-controller
$ tox -e integration -- --keep-models --model kubeflow
$ juju switch kubeflow
$ juju exec --unit grafana-agent-k8s/0 -- curl localhost:12345/agent/api/v1/metrics/targets | jq '.data.[] | select(.labels.juju_charm == "tensorboard-controller") | .endpoint'
"http://<IP_ADDRESS>:8080/metrics"
$ juju exec --unit grafana-agent-k8s/0 -- curl http://<IP_ADDRESS>:8080/metrics
# HELP controller_runtime_active_workers Number of currently used workers per controller
# TYPE controller_runtime_active_workers gauge
controller_runtime_active_workers{controller="tensorboard"} 0
# HELP controller_runtime_max_concurrent_reconciles Maximum number of concurrent reconciles per controller
# TYPE controller_runtime_max_concurrent_reconciles gauge
controller_runtime_max_concurrent_reconciles{controller="tensorboard"} 1

full metrics example
How to check alert rule
$ kubectl exec -it -n kubeflow pod/tensorboard-controller-0 -c tensorboard-controller -- /charm/bin/pebble stop tensorboard-controller
$  juju exec --unit grafana-agent-k8s/0 -- curl localhost:12345/agent/api/v1/metrics/targets | jq '.data.[] | select(.labels.juju_charm == "tensorboard-controller")
...
"scrape_error": "Get \"http://<IP_ADDRESS>:8080/metrics\": dial tcp <IP_ADDRESS>:8080: connect: connection refused"

After 5 minutes you should see alert firing (cos deployment is required).

fixes:#122

Using fork


java.lang.OutOfMemoryError in AnalyticsControllerImpl.java

Hi,
I added the latest version (1.0.0) of newrelic-react-native-agent and released a prod beta version of our app. Now I'm seeing the following issue  a lot on Android 13 in Pixel 5 and Pixel 6 Pro devices. If you need more info, please let me know. I'm more than happy to help out.
java.lang.OutOfMemoryError: Failed to allocate a 160614552 byte allocation with 97416504 free bytes and 92MB until OOM, target footprint 268435456, growth limit 268435456
	at java.util.Arrays.copyOf()(Arrays.java:3766)
	at java.lang.AbstractStringBuilder.ensureCapacityInternal()(AbstractStringBuilder.java:125)
	at java.lang.AbstractStringBuilder.append()(AbstractStringBuilder.java:650)
	at java.lang.StringBuilder.append()(StringBuilder.java:203)
	at java.util.AbstractMap.toString()(AbstractMap.java:559)
	at java.lang.String.valueOf()(String.java:3657)
	at java.lang.StringBuilder.append()(StringBuilder.java:132)
	at com.newrelic.agent.android.analytics.AnalyticsControllerImpl.recordCustomEvent()(AnalyticsControllerImpl.java:782)
	at com.newrelic.agent.android.NewRelic.recordCustomEvent()(NewRelic.java:1106)
	at com.newrelic.agent.android.NewRelic.recordCustomEvent()(NewRelic.java:1125)
	at com.NewRelic.NRMModularAgentModule.recordCustomEvent()(NRMModularAgentModule.java:215)
	at java.lang.reflect.Method.invoke()(Method.java:-2)
	at com.facebook.react.bridge.JavaMethodWrapper.invoke()(JavaMethodWrapper.java:372)
	at com.facebook.react.bridge.JavaModuleWrapper.invoke()(JavaModuleWrapper.java:188)
	at com.facebook.react.bridge.queue.NativeRunnable.run()(NativeRunnable.java:-2)
	at android.os.Handler.handleCallback()(Handler.java:942)
	at android.os.Handler.dispatchMessage()(Handler.java:99)
	at com.facebook.react.bridge.queue.MessageQueueThreadHandler.dispatchMessage()(MessageQueueThreadHandler.java:27)
	at android.os.Looper.loopOnce()(Looper.java:201)
	at android.os.Looper.loop()(Looper.java:288)
	at com.facebook.react.bridge.queue.MessageQueueThreadImpl$4.run()(MessageQueueThreadImpl.java:226)
	at java.lang.Thread.run()(Thread.java:1012)

I tested on many new and old devices and many android APIs, the app keeps crashing. I think probably the sdk is broken. Maybe, after enabling circular structure to JSON to fix the following issue, Android SDK cannot handle it.
https://github.com/newrelic/newrelic-react-native-agent/issues/64
Could you please have a look at this issue?
@kthai-nr

Hello, I've checked large structures on my side and everything continues to work fine. We have checks in place when recording these events and data larger than 4096 bytes will be dropped. So, I don't think the circular structure to JSON changes are causing this issue. It might be helpful to provide some Charles logs and a sample app where you can reproduce the issue if possible.

Hello @kthai-nr
This occurs only when the agent is active. Anyway, According to the following it is obvious this happens when the newrelic android agent is active
at com.newrelic.agent.android.analytics.AnalyticsControllerImpl.recordCustomEvent()(AnalyticsControllerImpl.java:782)
There is no way to create a sample app as our app is a very complex app and has many external packages installed  and I don't know which other dependencies are contributing to this error. Anyway I have attached charles proxy logs
CharlexProxyLogs.zip

Hello @kthai-nr
Adding to the above comment.
I can confirm, if you use getCircularReplacer function instead of the Crockford's cycle function in the fix that you made in this PR: https://github.com/newrelic/newrelic-react-native-agent/pull/65, this error doesn't happen.
I have attached CharlesProxy Logs after patching the package with the getCircularReplacer function.
3-with-Decycle-fn.zip

Hello @ladp333
Thanks for providing that information. I will revert this back to the previous implementation with getCircularReplacer in #73. I was able to reproduce an OOM error with:
    let arr = [];
    let prev = null;
    for(var i = 0; i < 1000000; ++i) {
      prev = arr[i] = {
        prev: prev
      }
    }
    NewRelic.sendConsole("log", arr);

It looks like reverting to getCircularReplacer worked fine on your Charles logs, but let me know if you are still running into issues after that since I don't know which dependency/library is causing this issue.

@ladp333 If you can test and confirm that this fixes the problem, I will push out a hotfix release for this. It is looking good on my end, but I'd like to double check.

@ladp333 If you can test and confirm that this fixes the problem, I will push out a hotfix release for this. It is looking good on my end, but I'd like to double check.

Thanks @kthai-nr . I tested again with getCircularReplacer by patching the pkg after this error happened and it looked good on iOS and Android. Just give me 30 mins. I will check double check your change.

@kthai-nr
It is looking good. Appreciate if you can push out a hotfix. Thanks

Thanks, appreciate your patience and diligence with this issue. I'll be pushing a hotfix shortly.

Hello @ladp333, we have pushed out release v1.0.1. You should be able to download it here: https://www.npmjs.com/package/newrelic-react-native-agent/v/1.0.1. Let me know if there are any other issues


Release Google.Cloud.Bigtable.V2 version 3.4.0

Changes in this release:
Bug fixes

Return all columns when multiple families have the same qualifier. (commit 137e40f)

New features

Add new_partitions field for CloseStream for Cloud Bigtable ChangeStream (commit 873b2fa)
Publish the Cloud Bigtable Change Streams (commit 5b4d425)

Created release for Google.Cloud.Bigtable.V2-3.4.0


[SUPPORT] Question on hudi's insert statment taking too long

Disclaimer
The situation is very similar to this one - https://github.com/apache/hudi/issues/3975 but I do not know they are the same or not.
Describe the problem you faced

There is a simple indexing hudi job - it reads parquet files (partitioned yyyy/mm/dd) and stored this data as is but with hudi indexes.
The job run in EMR c5.18xlarge (10 or 20 instances)
The job run per month. E.g. EMR cluster started and then each month processed via a separate step
There 2 data sources to process. Fiest one has input with about  8000+ objects and 1.1 TB data per month while the Second one has about 6000+ objects with 2.8 Tb data per month.
Hudi jobs completed without any errors for the first data source: 20 EC2 instances have >90% load. But the second job can not be finished. Sometimes it failed with OutOfMemoryError, sometimes it just holds. It was tried: 1) to switch to c5.24xlarge instances with about 1.5 times more memory than previous 2) to increase nodes up to 100
The first job completes with 4-5 hours (EC2 spot instances may increase the time)
While the second job took about 3 days without completing

Compare Graphs:

Are  there any idea why isEmpty at HoodieSparkSqlWriter.scala:609 operation may take so much time?
Feel free to ask any additional questions.
Thank you in advance!
Environment Description

Hudi version : 0.9.9

Spark version : 2.4.4

Storage (HDFS/S3/GCS..) : S3

Running on Docker? (yes/no) : no

@nikita-sheremet-clearscale could you share the hudi configs you used for this job? e.g. bulk insert or upsert? COW or MOR table? any config you set manually can be helpful.
From the screenshot, it looks like it's doing a rollback. probably the first job did not finish properly. can you also check the hoodie timeline in .hoodie/ see if the last commit completed or not.

Many thanks for the quick reply!!!
Hudi config is:
hoodie.datasource.hive_sync.database -> hudi
hoodie.datasource.write.row.writer.enable -> false
hoodie.parquet.small.file.limit -> 104857600
hoodie.datasource.hive_sync.partition_fields -> y,m,d
hoodie.bloom.index.filter.dynamic.max.entries -> 100000000
hoodie.datasource.hive_sync.partition_extractor_class -> org.apache.hudi.hive.MultiPartKeysValueExtractor
hoodie.parquet.max.file.size -> 209715200
hoodie.parquet.page.size -> 1048576
hoodie.bloom.index.prune.by.ranges -> false
hoodie.parquet.block.size -> 209715200
hoodie.datasource.hive_sync.table -> table
hoodie.index.type -> BLOOM
hoodie.datasource.write.operation -> bulk_insert
hoodie.datasource.hive_sync.enable -> true
hoodie.datasource.write.recordkey.field -> device_uuid
hoodie.table.name -> table
hoodie.datasource.write.table.type -> COPY_ON_WRITE
hoodie.datasource.write.hive_style_partitioning -> true
hoodie.bulkinsert.sort.mode -> NONE
hoodie.datasource.write.keygenerator.class -> org.apache.hudi.keygen.ComplexKeyGenerator
hoodie.datasource.write.drop.partition.columns -> true
hoodie.datasource.write.partitionpath.field -> y,m,d
hoodie.bloom.index.filter.type -> DYNAMIC_V0
hoodie.bulkinsert.shuffle.parallelism -> 15000
hoodie.index.bloom.num_entries -> 100000

The file<PHONE_NUMBER>3538.rollback contains the following:
Objavro.schema�{"type":"record","name":"HoodieRollbackMetadata","namespace":"org.apache.hudi.avro.model","fields":[{"name":"startRollbackTime","type":{"type":"string","avro.java.string":"String"}},{"name":"timeTakenInMillis","type":"long"},{"name":"totalFilesDeleted","type":"int"},{"name":"commitsRollback","type":{"type":"array","items":{"type":"string","avro.java.string":"String"}}},{"name":"partitionMetadata","type":{"type":"map","values":{"type":"record","name":"HoodieRollbackPartitionMetadata","fields":[{"name":"partitionPath","type":{"type":"string","avro.java.string":"String"}},{"name":"successDeleteFiles","type":{"type":"array","items":{"type":"string","avro.java.string":"String"}}},{"name":"failedDeleteFiles","type":{"type":"array","items":{"type":"string","avro.java.string":"String"}}},{"name":"rollbackLogFiles","type":["null",{"type":"map","values":"long","avro.java.string":"String"}],"default":null},{"name":"writtenLogFiles","type":["null",{"type":"map","values":"long","avro.java.string":"String"}],"default":null}]},"avro.java.string":"String"}},{"name":"version","type":["int","null"],"default":1},{"name":"instantsRollback","type":{"type":"array","items":{"type":"record","name":"HoodieInstantInfo","fields":[{"name":"commitTime","type":{"type":"string","avro.java.string":"String"}},{"name":"action","type":{"type":"string","avro.java.string":"String"}}]},"default":[]},"default":[]}]} �H�y�ܘ�3��M��_<PHONE_NUMBER>3538���G<PHONE_NUMBER>0330 � y=2021/m=04/d=30 y=2021/m=04/d=30     y=2020/m=06/d=30 y=2020/m=06/d=30     y=2020/m=11/d=30 y=2020/m=11/d=30     y=2020/m=11/d=23 y=2020/m=11/d=23     y=2020/m=11/d=24 y=2020/m=11/d=24     y=2020/m=11/d=21 y=2020/m=11/d=21     y=2020/m=11/d=22 y=2020/m=11/d=22     y=2020/m=11/d=27 y=2020/m=11/d=27     y=2020/m=11/d=28 y=2020/m=11/d=28     y=2020/m=11/d=25 y=2020/m=11/d=25     y=2020/m=11/d=26 y=2020/m=11/d=26     y=2020/m=11/d=20 y=2020/m=11/d=20     y=2020/m=11/d=29

Then there is a list of partition key and file on s3. Why rollback operation is performed on so many partitions? Only partition with y=2021/m=07 was run.
Btw hudi have a data and table. What happens when some partitions were deleted and then indexed?

Btw is there documentation how hudi handles interrupted data? Like there was EMR job that failed/stopped in the middle. Then data created by this job were deleted manually and reinserted again. (No hudi delete or upsert operations).

@xushiyan
I have rn hudi-cli.sh show rollbacks the outout is:
hudi:truedata_detections->show rollbacks
21/11/21 18:13:53 INFO timeline.HoodieActiveTimeline: Loaded instants [[20210606040926__rollback__COMPLETED], [20210713074934__rollback__COMPLETED], [20210719111122__rollback__COMPLETED], [20211028080814__rollback__COMPLETED], [20211028080841__rollback__COMPLETED], [20211028080857__rollback__COMPLETED], [20211102154040__rollback__COMPLETED], [20211102185934__rollback__COMPLETED], [20211104113903__rollback__COMPLETED], [20211108130332__rollback__COMPLETED], [20211115193538__rollback__COMPLETED], [20211116071046__rollback__COMPLETED], [20211116175746__rollback__COMPLETED], [20211117072205__rollback__COMPLETED], [20211119122136__rollback__COMPLETED]]
21/11/21 18:13:53 INFO s3n.S3NativeFileSystem: Opening 's3://bucket/table/.hoodie/20210606040926.rollback' for reading
21/11/21 18:13:54 INFO s3n.S3NativeFileSystem: Opening 's3://bucket/table/.hoodie/20210713074934.rollback' for reading
21/11/21 18:13:54 INFO s3n.S3NativeFileSystem: Opening 's3://bucket/table/.hoodie/20210719111122.rollback' for reading
21/11/21 18:13:54 INFO s3n.S3NativeFileSystem: Opening 's3://bucket/table/.hoodie/20211028080814.rollback' for reading
21/11/21 18:13:54 INFO s3n.S3NativeFileSystem: Opening 's3://bucket/table/.hoodie/20211028080841.rollback' for reading
21/11/21 18:13:54 INFO s3n.S3NativeFileSystem: Opening 's3://bucket/table/.hoodie/20211028080857.rollback' for reading
21/11/21 18:13:54 INFO s3n.S3NativeFileSystem: Opening 's3://bucket/table/.hoodie/20211102154040.rollback' for reading
21/11/21 18:13:54 INFO s3n.S3NativeFileSystem: Opening 's3://bucket/table/.hoodie/20211102185934.rollback' for reading
21/11/21 18:13:54 INFO s3n.S3NativeFileSystem: Opening 's3://bucket/table/.hoodie/20211104113903.rollback' for reading
21/11/21 18:13:55 INFO s3n.S3NativeFileSystem: Opening 's3://bucket/table/.hoodie/20211108130332.rollback' for reading
21/11/21 18:13:55 INFO s3n.S3NativeFileSystem: Opening 's3://bucket/table/.hoodie/20211115193538.rollback' for reading
21/11/21 18:13:55 INFO s3n.S3NativeFileSystem: Opening 's3://bucket/table/.hoodie/20211116071046.rollback' for reading
21/11/21 18:13:55 INFO s3n.S3NativeFileSystem: Opening 's3://bucket/table/.hoodie/20211116175746.rollback' for reading
21/11/21 18:13:55 INFO s3n.S3NativeFileSystem: Opening 's3://bucket/table/.hoodie/20211117072205.rollback' for reading
21/11/21 18:13:55 INFO s3n.S3NativeFileSystem: Opening 's3://bucket/table/.hoodie/20211119122136.rollback' for reading

╔═════════════════════╤════════════════════╤═════════════════════╤══════════════════════╤══════════════════╗
║ Instant             │ Rolledback Instant │ Total Files Deleted │ Time taken in millis │ Total Partitions ║
╠═════════════════════╪════════════════════╪═════════════════════╪══════════════════════╪══════════════════╣
║ 2021-06-06T04:09:26 │<PHONE_NUMBER>3604     │ 9762                │ 38399                │ 153              ║
╟─────────────────────┼────────────────────┼─────────────────────┼──────────────────────┼──────────────────╢
║ 2021-07-13T07:49:34 │<PHONE_NUMBER>5041     │ 0                   │ 8163                 │ 481              ║
╟─────────────────────┼────────────────────┼─────────────────────┼──────────────────────┼──────────────────╢
║ 2021-07-19T11:11:22 │<PHONE_NUMBER>2859     │ 0                   │ 10832                │ 481              ║
╟─────────────────────┼────────────────────┼─────────────────────┼──────────────────────┼──────────────────╢
║ 2021-10-28T08:08:14 │<PHONE_NUMBER>0928     │ 0                   │ 8265                 │ 275              ║
╟─────────────────────┼────────────────────┼─────────────────────┼──────────────────────┼──────────────────╢
║ 2021-10-28T08:08:41 │<PHONE_NUMBER>0812     │ 0                   │ 7521                 │ 276              ║
╟─────────────────────┼────────────────────┼─────────────────────┼──────────────────────┼──────────────────╢
║ 2021-10-28T08:08:57 │<PHONE_NUMBER>0840     │ 0                   │ 8094                 │ 276              ║
╟─────────────────────┼────────────────────┼─────────────────────┼──────────────────────┼──────────────────╢
║ 2021-11-02T15:40:40 │<PHONE_NUMBER>4711     │ 0                   │ 11370                │ 396              ║
╟─────────────────────┼────────────────────┼─────────────────────┼──────────────────────┼──────────────────╢
║ 2021-11-02T18:59:34 │<PHONE_NUMBER>4038     │ 0                   │ 11588                │ 396              ║
╟─────────────────────┼────────────────────┼─────────────────────┼──────────────────────┼──────────────────╢
║ 2021-11-04T11:39:03 │<PHONE_NUMBER>5933     │ 0                   │ 10932                │ 396              ║
╟─────────────────────┼────────────────────┼─────────────────────┼──────────────────────┼──────────────────╢
║ 2021-11-08T13:03:32 │<PHONE_NUMBER>2529     │ 27723               │ 259824               │ 457              ║
╚═════════════════════╧════════════════════╧═════════════════════╧══════════════════════╧══════════════════╝

Hey @nikita-sheremet-clearscale : what do you mean by "What happens when some partitions were deleted and then indexed?" ? can you please clarify. In general, its not encouraged to delete data files manually, but leave it to rollback to take care of cleaning up partially failed commits.
my guess is that, your large job failed due to OOM or some other reason. And so the next job you started, does a rollback of the partially failed data which is too huge and hence takes a lot of time. but even then, deletions should not take such a long time though.

@nsivabalan
Many thanks for reply! :-)
What happens when some partitions were deleted and then indexed?
It means that I got to S3 web console, select prefixes and click delete button. I understand that it is not the right way but I have restrictions on code modification so I can not change jobs to cleanup all old data when data are rewritten.
my guess is that, your large job failed due to OOM or some other reason. And so the next job you started, does a rollback of the partially failed data which is too huge and hence takes a lot of time. but even then, deletions should not take such a long time though.
Can this happen due to EC2 spot instance termination?

we made a perf fix around fixing the issue you have reported.
https://github.com/apache/hudi/pull/5187
this should be available from 0.11. But in general, we also made few critical perf fixes with hudi in 0.12. So, would recommend to upgrade to latest hudi version.
thanks!

feel free to open a new issue if you are looking for further assitance,


When initializing flatpickr with onclose function, the console displays "mode of undifined" error

When initializing flatpickr with onclose function, the console displays "mode of undifined" error.
example,
flatpickrInit: function () {
	// initialize flatpickr
	var flat = $('#myID').flatpickr({
		mode: 'range',
		dateFormat: 'Ymd',
		onClose: function (seldates, datestr) {
				self.flat = self.flatpickrInit();
			}
		}
	});
}

Hi @kyken,
Can you please provide an working example where we can see this error? Please fork this fiddle https://jsfiddle.net/tj2Lo4z7/
Without that, I can't say for sure. I could take a guess that it's because you're reinstantiating flatpickr everytime you close the calendar. In this case, it has nothing to do with onClose, but how you use it.

Closing this due to inactivity. Please feel free to reopen if the problem persists, including a working example showcasing the problem


define and describe debug.log(keys) format

ref. https://jimshaver.net/2015/02/11/decrypting-tls-browser-traffic-with-wireshark-the-easy-way/

About the format of The key log file
ref.https://www.trustwave.com/Resources/SpiderLabs-Blog/How-To-Decrypt-Ruby-SSL-Communications-with-Wireshark/

The general format is " ".

We should follow the following rough format guidelines.
id: First 18 bytes of encrypted packet ???
key: A sending key and a nonce ???

Done https://github.com/nayutaco/lightning-dissector/blob/master/CONTRIBUTING.md


fix: only run DOM operations in the client when there is a document (#17570)

Description
Fixes #17570

I ran prettier and created another commit but the linter still fails. Not sure what to do.

I ran prettier and created another commit but the linter still fails. Not sure what to do.

Probably you need to run pnpm i before running pnpm format.

Do you mean we would display a warning? Or that the code should be skipped? I'm not sure how a warning would make sense, as there is no CSS in a worker, so it's not something that you would want to hear about not happening.

I mean displaying a warning. There's no CSS in a worker and the CSS import won't work, so I think a warning should be there. Why do you think the warning shouldn't be happening? (I am trying to understand your thoughts.)

The way I see it is that the client is just vite injected code that I have no control over as a developer. I am just trying to write code, and I don't even "care" that vite injects that code into my browser and worker modules.
The whole reason I started this PR is because I was getting errors in the console. It is not obvious that these are worker errors from dev tools. My thought is that adding other warnings would just cause confusion because seeing something like "CSS could not load in a worker" would not make sense because I am not trying to load CSS in a worker consciously. Maybe I happen to import a file that also imports CSS, but since CSS doesn't work in that context, it would just not be loaded.
In other words, what would I do with that warning, other than be slightly confused where it is coming from?
I would suggest that we keep each PR focused. This one fixes a bug: throwing uncaught errors in workers (which can cause real problems in workers). If there is a desire to add more warnings around things, or to refactor to be more worker-conscious, that could happen in a follow-up PR as a feature not a bug.

I agree that the client is not something that developers would / should care about. But the CSS import means appending that CSS into the document and if that is not desired the developer should remove that import. I think this is same situation when you have a code that depends on document (e.g. document.title = 'title'), in this case you have to remove that code. The warning will tell the developers to remove the CSS import.
If you want to trim down the scope much more, I'd say let's only changing the error message and don't ignore the error. If we're ignoring the developer's code error, IMO we should have a warning message. But I'll let others chime in.

I don't have enough insight into vite to make a call on this. All I know is that the error messages I fixed are ones that are showing up as a result of errors in the client and had nothing to do with my code. I started this PR because the client was trying to look for its own overlays, which causes errors. That is why I'm suggesting that we separate "don't let vite client break" from "find the right amount of warnings."

In other words, this addresses #17570. If we want to address warnings, I'd prefer that be a separate issue with a separate PR. I don't have the time to contribute more than this fix anyway, so either this goes stale and the bug persists, or there can be a separate issue on the back log.


Mumble-voip

Hey, was looking to give it a shot but I ended up not even able to start it having an issue with depedencie
"Could not find dependency mumble-voip for resource rp-radio." when starting I'm running my server on Debian so a Linux release of FiveM server might be the issue?

I'm dumb.


Replace func.func/call with util.func/call.

Finally time. We need functions that support tied operands and inlining control as today func.func has no good way to do it. We don't need shape/size-aware operands/results as IPO takes care of that in general and we don't want to require result sizes to be computable by callers.
When we have our own version we can control inlining; today the inliner control is handled by the func dialect so we can't do alwaysinline/noinline, can't avoid inlining into different device-scoped contexts, and can't add our own cost modeling for whether inlining is profitable.
We don't support function addresses today so we can ignore those ops and just bring over func.func/func.call as util.func/util.call.

[ ] identify places that can utilize tied operands (allocation, IPO, refinement, etc)
[ ] new pass (or in IPO?) to tie operands on funcs based on all call sites
[x] convert to util.func as part of the input pipeline (and update all tests)
[ ] implement inlining policy on hal.device.targets/stream.affinity to validate interface

Most of the func.func->util.func changes landed as part of #16411. There's still work to do to make use of the new util.func tied operand feature and enable IPO to work with it but that will come later.


[wallet/desktop] fix: removed duplicate message property from en-US.json

What was the issue?
There is a duplicate message property in en-US.json. Reported https://github.com/symbol/desktop-wallet/issues/1961
The chosen name "Reclaim" is as this transaction type is presented in explorer.
What's the fix?
Removed duplicate message property.

Codecov Report

Merging #1962 (bce8f04) into dev (e0f89c2) will increase coverage by 0.00%.
The diff coverage is n/a.

Additional details and impacted files

@@           Coverage Diff           @@
##              dev    #1962   +/-   ##
=======================================
  Coverage   53.62%   53.63%
=======================================
  Files         584      584
  Lines       14004    14004
  Branches     2348     2348
=======================================
+ Hits         7510     7511    +1
+ Misses       6444     6443    -1
  Partials       50       50

Flag
Coverage Δ

wallet-desktop
53.63% <ø> (+<0.01%)
:arrow_up:

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files
Coverage Δ

src/core/utils/TimeHelpers.ts
69.79% <0.00%> (+1.04%)
:arrow_up:


[INTEGRATION][Spark] provide physical location of a table facet in case we're using logical table name

When dealing with a logical Hive table, we've decided to use logical Hive table name as a dataset name. In some other cases, some other job - not necessarily Spark one - might deal with this dataset by directly knowing it's physical location. In this case we'll have the same dataset known by two different names in different namespaces.
To make sure those datasets can be "linked" by OpenLineage consumers, when available, we should provide additional facet (PhysicalLocationFacet?) that indicates physical location at a storage system (like hdfs, s3 etc) or other underlying storage mechanism.
Make sure to update Naming.md to standarize how we're naming hive dataset.

This is a great idea and would be very helpful for our work integrating OpenLineage with Apache Atlas.

One could also consider extending StorageDatasetFacet to contain a field with such an information -> https://github.com/OpenLineage/OpenLineage/blob/main/spec/facets/StorageDatasetFacet.json

I have lot of customers using Delta Table on Spark (Databricks), we met the challenge that the OpenLineage might not be able to support SQL table name. May I draw your immediate attention please to consider prioritize this enhancement, thanks.

@mobuchowski - In what cases does OpenLineage currently report the hive table name?
I couldn't find an example in the unit / integration tests and the Naming guide does not seem to include Hive.
spark.sql("SELECT * FROM tableName")  appears to return the physical path rather than the Hive table name. Is that expected or am I just getting a different experience because I'm using Databricks?

@wjohnson I think this is also about changing that 😉 a valid point though  - modified issue to indicate that.
@georgeliang0821 this issue shouldn't modify anything regarding Delta Lake AFAIK, just hive for now

Also, as you see, this is not a fresh issue - it was made around time when we were filling gaps in Hive coverage with @pawel-big-lebowski - would be easier to do back then rather than now when you're using it already 😞

I know there's a PR open for this, but I'm wondering whether it is better to include the physical location as a facet or the Hive table information. Assuming a dataset is read outside of Hive (e.g., as in a Hive External Table), the physical location means lineage is immediately established with no work on the server side. If, however, we store physical location as a facet, the server has to know to look at that facet and determine whether there is already an existing dataset at that location.

I know there's a PR open for this, but I'm wondering whether it is better to include the physical location as a facet or the Hive table information. Assuming a dataset is read outside of Hive (e.g., as in a Hive External Table), the physical location means lineage is immediately established with no work on the server side. If, however, we store physical location as a facet, the server has to know to look at that facet and determine whether there is already an existing dataset at that location.

I think that's the behavior I'd want too.  Ideally, we could link to the table identifier but still have the physical file path.

Query: SELECT * FROM table1
If tabl1 is a managed table

name: table1
namespace /usr/hive/warehouse

If table1 is an external table

name: 'someFoldername`
namespace: (abfss|s3|wasb|file|dbfs):/path/to/
facet: TableIdentifier: default.table1

Let me sum up my understanding of the discussion above. Some folks may prefer to identify datasets by its physical location, some may want to use logical / relational identifiers like database & table.
We cannot please them all as we do have just two fields: name & namespace. There are two possible solutions:

Identify datasets by physical location.

Introduce extra facet (CatalogIdentifierFacet) to contain catalog/datbase/table.
Modify existing behaviour of the Spark integration for V2 relations to have identifier containing a path instead of table name as currently. Include CatalogIdentifierFacet in OL event.
No extra work on backend is required to link datasets.
After a version switch, users may experience having the same datasets identified in a different way.

Introduce PhysicalLocationFacet and don't change the current behaviour and

naming convention will stay insonsistent but won't get modified (possible mess in Marquez for current users)
PhysicalLocationFacet will be always sent

My initial approach was to go with 2. in order not to change the existing behavior.
@wjohnson  and @collado-mike  are opting for 1. Right?
What about @mobuchowski ?

Let's first describe the currently implemented algorithm to obtain DatasetIdentifier. There are two options:

If a table is accessed with V2 table API, then DatasetIdentifer is evaluated within one of three catalog handlers: 'IcebergHandler', 'DeltaHandler' or 'DatabricksDeltaHandler' (which is pretty similar to 'DeltaHandler'). Each handler implements a method:

 public DatasetIdentifier getDatasetIdentifier(
      SparkSession session,
      TableCatalog tableCatalog,
      Identifier identifier,
      Map<String, String> properties)

Delta handlers always return a location path of a table. IcebergHandler returns different results based on the spark.sql.catalog.<catalogName>.type which can be hive or hadoop. In case of hadoop location path is used to create DatasetIdentifier. In case of hive, it is checked if a config field spark.sql.catalog.uri is defined. If both conditions are met, then DatasetIdentifier containing table name is returned  like {"name": "database.schema.table", "namespace": "hive://metastore-host:10001"}).

For non v2 tables, methods in PathUtils are used to obtain DatasetIdentifier. Some of them like fromPath or fromUri return location identifier but there is a method fromCatalogTable which verifies if hive support is enabled and if config entry spark.sql.hive.metastore.uris is set. If so, metastore uri enriched with table qualifed name is returned.

To sum up a database represantation identifier is returned only in two scenarios:

iceberg table with hive catalog type,
hive enabled Spark session with external metastore uris defined in config.

My initial approach was to add physicalLocation facet everywhere which does not change the existing behavior of the code, but requires linking datasets on the backend side and may be overkill as most of the time physicalLocation is equal to DatasetIdentifier
@wjohnson  and @collado-mike are opting for the approach where we replace existing behavior of the two scenarios with location dataset identifier and introduce extra facet (like TableIdentifierFacet) which will contain database and table names (right?). If we are fine with modifying existing behavior, this approach looks cleaner to me either.

@wjohnson and @collado-mike are opting for the approach where we replace existing behavior of the two scenarios with location dataset identifier and introduce extra facet (like TableIdentifierFacet) which will contain database and table names (right?). If we are fine with modifying existing behavior, this approach looks cleaner to me either.

Thank you for this excellent summary, Pawel!  A good analysis like this belongs in the docs page!
You described exactly what I was looking for in this last section.  I just don't know how to accomplish it.

@wjohnson looked at this again, and my opinion has changed.
I think we also need to take this discussion outside of just Spark context 😉
Why? Because we also want to support them from other integrations, like Flink integration support for Iceberg.
We generally have three types of systems:

Ones that refer to their datasets by logical names - for example, the traditional relational databases which follow the schema dataset.table.name - sometimes changing it by dropping schema like MySQL for example.
Others refer to the datasets by physical locations - for example
There is also third type, which can have both logical and physical names. For example Delta Lake and Iceberg.

In that case, we need to take a look what actual system does: for example, Delta supports two table types but actually clearly warns against relying just on the catalog:
The metastore is not the source of truth about the latest information of a Delta table. In fact, the table definition in the metastore may not contain all the metadata like schema and properties.
It contains the location of the table, and the table’s transaction log at the location is the source of truth.
If you query the metastore from a system that is not aware of this Delta-specific customization, you may see incomplete or stale table information.

Iceberg in the face always refers to the datasets as catalog.schema.table and requires using catalogs when using Spark SQL, but the catalogs aren't that important too - multiple ones can point to the same location and always having catalog means you can consistently manage metadata without different table types.
In those systems the actual location is most important and it's what defines dataset, so we definitely should refer to it as actual dataset namespace and name. Any logical name should be an additional info. So @pawel-big-lebowski is definitely right in the last paragraph, but we should migrate from database represantation identifier in Iceberg/Hive and just Hive if doable.
Now, regarding databricks... it's more complex?
Looks like it just adds layer on top of Spark complexity with Unity Catalog. I'm not sure if we need to do anything with it yet, and that Spark based approach we have today will work.


test runner - printed VALID code is INVALID

The test runner claim that valid printed code is invalid. This is a HUGE problem for me. Currently stuck with over 4000 files with this issue.
Input
class A { foo(a, b,) {} };

Printed
class A {
  foo(
    a,
    b,
  ){}
}

I can add to this issue that it sometimes it "feels like" I'm dealing with two different parsers. Same settings and same parser in a local test environment as used in the test runner output different CST so hard to track where the issues is.

maybe it's the parser's issue? the message means printed code diagnostics's length not equal with the input's.
https://github.com/kataw/kataw/blob/496f8e3f513d74b927200827aa94a8d7e3590e8b/test/runner/tob.ts#L65

How can it be a parser issue? I'm also verify the output manually against the parser. No issue

If it's the parser then it brings me to the second discovery  - that it feels sometimes I'm dealing with two parses. See my comment above

how can I repro the issue? did you push the change?

I can't push it, but it sometimes happen with current code online. The printer. For example the printer - through the test runner - accepted to print this invalid code - a?.?.b.  It was not caught at all.
see one of my latest commit to the repo to throw error for that case.
Here the 'chain' property is null because it's invalid. The test runner didn't pick it up. It should be invalid to reparse it.

And my current Kataw code locale I can't even push it to Kataw repo because it will break. The tests will fail. But locale it they all pass and no errors.

can you push it to another branch?

What to push? I gave an example in first comment, and explained how it printed invalid code.

The code I'm playing with locale catch all this issues that's why I report them and found this test runner issue. Just now the test runner generated a 100 MB file that I was forced to delete.

@aladdin-add What do you want? Do you want the 'the best"? Wait!

"best"?
please push your branch to the gh, so I was able to repro it!

"The beast" is the name I use on my private printer. It's an Prettier equal code that I play around with.
It's incoming!

it was working on my local, too. I will create a PR to test it.

Also change the 'tob.ts' back to use the original printer and it breaks tests even the code is correct.

@aladdin-add Here is how you can reproduce

Use 'the beast' and print the code with 'allowArrowParens' enabled

It should transform 'async x=>x, y' into 'async (x)=>x, y', but nothing happens and it report that 'async x=>x, y' is an invalid code!
Turn off the option and suddenly 'async x=>x, y' is valid code!

if you try to re-parse the printed code, you will get Declaration or statement expected. it's why it was treated as invalid.
I think we just need to improve the error message.

Can you improve the error message? I will have a look at the parser code for async

Closing as the mention option works now. It was an error with async arrow parsing. Fixed!


Fix areaOfTriangle method

There is currently a method called areaOfTriangle that lives in /lib/areaOfTriangle.js.
It is incomplete and needs to be fixed!
Acceptance Criteria:

Running yarn test areaOfTriangle results in tests passing.
You have written a skeleton method for someone else to work on.
You have written tests surrounding your skeleton method.
Running yarn lint does not print any errors to the console!
Optional: write one or two more tests surrounding the method you are solving to account for potential edge cases.

Please include the skeleton of a new method + an accompanying test for someone else to work on at the time of creating a pull request! A pull request will most likely be denied if it does not contain a skeleton method for someone else to work on! For more information, please read the Contributing Guide.
Thank you so much for your contribution!

I'll take this one.

Sorry didn't realize someone already took it, go ahead.

thanks @kurnifamili , i'll start working on it.


Infer good object depth for initial 3D lattice config

Less this please:

Object depth should also be tracked as a dependency when recalculating the view-box.


Add files via upload

@youhou974 stcharles.txt: this request has an incorrect file path format, please ensure that you are familiar with     this repository rules located at the bottom of this page and change     your request accordingly. If you don't know how to change request, close it and open a new one.     If you are not able to make the pull request according the instructions, please send us an email to<EMAIL_ADDRESS>Please mention this request URL in the message.

@youhou974 stcharles.txt: this request has an incorrect file path format, please ensure that you are familiar with     this repository rules located at the bottom of this page and change     your request accordingly. If you don't know how to change request, close it and open a new one.     If you are not able to make the pull request according the instructions, please send us an email to<EMAIL_ADDRESS>Please mention this request URL in the message.

@youhou974 stcharles.txt: this request has an incorrect file path format, please ensure that you are familiar with     this repository rules located at the bottom of this page and change     your request accordingly. If you don't know how to change request, close it and open a new one.     If you are not able to make the pull request according the instructions, please send us an email to<EMAIL_ADDRESS>Please mention this request URL in the message.

@youhou974 stcharles.txt: this request has an incorrect file path format, please ensure that you are familiar with     this repository rules located at the bottom of this page and change     your request accordingly. If you don't know how to change request, close it and open a new one.     If you are not able to make the pull request according the instructions, please send us an email to<EMAIL_ADDRESS>Please mention this request URL in the message.

@youhou974 I am sorry, this request remains unanswered for a while. I have to close it without merging. Please reopen and comment if you have an update regarding the aforementioned issue. Thank you for the understanding!

Hello, I added the file as it was in the directories to add my school. Can you help me correct what is wrong please ?


Set aria-label as static value

Description
Implementing an aria-label which doesn’t change its value may help with guaranteeing the label will be read correctly for a variety of assistive technology. Having the aria-label update dynamically, paired with the aria-live attribute may not have the best support.
For example, JAWS paired with Edge browser does not announce the updated aria-label as expected. Chrome for Android announces the position of the stage, not what the model contains.
Recommendation: set the aria-label value as, “A 3D model of an astronaut” by default in order to guarantee this label announcement no matter where the user navigates from. Do not change this value throughout the experience.
<canvas … aria-label="A 3D model of an astronaut">
  <!-- … -->
</canvas>

Live Demo

Go to <model-viewer> with Chrome for Android
Load up TalkBack screen reader
Swipe right with one finger until the canvas content is found

Browser Affected

[x] Chrome
[x] Edge
[ ] Firefox
[ ] Helios
[ ] IE
[ ] Safari

OS

[x] Android
[ ] iOS
[ ] Linux
[ ] MacOS
[x] Windows

Versions

model-viewer: v0.1.1

@svinkle Thanks for this feedback. Are you proposing that live updates should no longer happen altogether, or just that we should find an alternative strategy to make them work?

I'm proposing keeping the aria-label as static text, and adjusting the aria-live treatment as described in #461.

Ahh, yes, could you perhaps try again with the latest build on master? I discovered a bug related to what you are describing while working on #452 and added a test for the preferred behavior.


[uperf][enhancement] Migrate to bulk indexer

Current indexing code is extremely trivial and can drop results. We need to migrate to the bulk indexer with retries ASAP.

Already migrated


[php] fix GPU

All the functions are there (if you're using my custom php-sdl branch)... but for some reason the window doesn't show up

Got the window opening now - but rendering to a texture shows up corrupt (rendering direct to hardware works ok)


use of genericModelSubstitutes

Hi,
If I do this:
@RequestMapping(value = "/test3", method = RequestMethod.GET)
@ResponseBody
public ResponseEntity<Resource<Album>> test3() {
         return null;
}

configured with
@Bean
public SwaggerSpringMvcPlugin swaggerSpringMvcPlugin(SpringSwaggerConfig springSwaggerConfig) {
        return new SwaggerSpringMvcPlugin(springSwaggerConfig)
.genericModelSubstitutes(ResponseEntity.class, Resource.class);
}

the json is:
"path": "/music/test3",
        "description": "test3",
        "operations": [{
            "method": "GET",
            "summary": "test3",
            "notes": "test3",
            "type": "Resource«Album»",
            "nickname": "test3",
            "parameters": [],

"type": "Resource«Album»", I was expecting to have Album because Resource is also a genericModelSubstitute
So in the UI (http://localhost:9080/spring3-music/sdoc.jsp#!/albums/test3) there is no "Model" associated.
You can test here: https://github.com/ctamisier/swagger-springmvc-demo/commit/0bf4ead2b9fff6c538d6ea806646ed74cfa01232
could it be a bug ?
Thanks for your help.

I'm using version 2.4.0 and a List<Resource<Article>> with Resource.class being added as genericModelSubstitutes outputs:
200: {description: "OK", schema: {type: "array", items: {$ref: "#/definitions/Resource«Article»"}}

But another endpoint returning Resource<Article> outputs:
200: {description: "OK", schema: {$ref: "#/definitions/Article"}}

Am I missing something?

Would you mind trying with 2.6.1. Pretty sure this bug is fixed @rlogiacco

@dilipkrish I updated to 2.6.1 and also added a forCodeGeneration(true) just to ensure the update went through correctly (you never know) and I now get:
200: {description: "OK", schema: {type: "array", items: {$ref: "#/definitions/ResourceOfAgreement"}}}

Apparently the bug is still there in my case.
What I noticed trying to debug the thing is that List seems not being returning true when hasWildcards(...) is executed... I'm looking at a breakpoint placed at line 49 of springfox.documentation.schema.AlternateTypeRule.alternateFor(ResolvedType)...
In other words it seems the problem, in this case, lies on the outer List element which doesn't allow the resolution to start... Any advice?

Oh yeah I forgot. You need to add an alternate type rule for that explicitly. A rule that replaces List<Resource with List.

newRule(
    resolver.resolve(List.class, resolver.resolve(Resource.class, WildcardType.class)),
    resolver.resolve(List.class, WildcardType.class));


[R] Custom binding documentation compare_dplyr_binding  function reference

Describe the usage question you have. Please include as many useful details as  possible.
On the Arrow 13 pkgdown site, the writing bindings documentation references a compare_dplyr_binding function (see code block below). From which package can this function be found?
test_that("startsWith behaves identically in dplyr and Arrow", {
  df <- tibble(x = c("Foo", "bar", "baz", "qux"))
  compare_dplyr_binding(
    .input %>%
        filter(startsWith(x, "b")) %>%
        collect(),
    df
  )
})

Arrow documentation: https://arrow.apache.org/docs/dev/r/articles/developers/writing_bindings.html
Rmd on GitHub: https://github.com/apache/arrow/blob/main/r/vignettes/developers/writing_bindings.Rmd
Component(s)
R

We might want to add a mention of this file to that documentation directly (and would very much welcome a PR that does that!) but that function is contained in our testthat helpers:
https://github.com/apache/arrow/blob/3db62050daa67b927810574dea60fd5b84bcb523/r/tests/testthat/helper-expectation.R#L70-L103
If you're using devtools or testthat to run tests it will be available while the tests are running (but it isn't contained in the packaged code since it's generally not necessary for end users)

This makes a lot of sense @jonkeane, thanks for the clarification on all of this. I'll make a PR on the documentation as your suggested, thanks again.


cursor in wrong position in a regular buffer displayed in a popup terminal

Describe the bug
After pressing gf on a file path in a popup terminal, the cursor is wrongly positioned whenever insert or visual mode is entered.
To Reproduce
Write this in /tmp/test.vim:
set hidden
let bufnr = term_start(&shell, #{hidden: v:true})
let winid = popup_create(bufnr, {'minwidth': &columns/2, 'border': [], 'minheight': &lines/2})
call feedkeys('/tmp/file')

Run this shell command:
printf -- 'foo\nbar' >/tmp/file && vim -Nu NONE -S /tmp/test.vim

Press C-\ C-n to switch to Terminal-Normal mode.
Press gf to load the file whose path is under the cursor.
Press V to select the line foo; Vim seems to ignore the first character f, and the cursor jumps to the next line.
Keep pressing j to extend the visual selection down to the lines below; the cursor moves to lines which don't contain text, it then goes beyond the popup and only stops once it reaches the bottom-right corner of the terminal window.

Expected behavior
One of these statements is true:

It is forbidden to load a regular buffer in a popup terminal, and gf/gF raise E994 (just like C-w f) or E863 (just like :e /tmp/file).

It is allowed for gf/gF to load a regular buffer in a popup terminal, in which case:
Pressing V initially selects the whole line foo, and the cursor stays on the latter.
Pressing j repeatedly selects the lines foo and bar; the cursor does not go beyond those.

Screenshots

Environment

Vim version: 8.2 Included patches: 1-740
OS: Ubuntu 16.04.6 LTS
Terminal: XTerm(322)

Additional context
There are similar issues when entering insert mode.
Sometimes, the cursor and the character which is inserted are in different positions.

The issue seems to disappear if the popup is created without a border.

Looks like you found a way to escape the terminal in the popup.  That should not be possible.  I think it should not even be possible to enter Terminal-Normal mode.  Let's see if I can disable that.

Hmm, a test is actually using Terminal-Normal mode, thus we need to disallow editing another file.


Fix broken docs.microsoft.com link

PR Summary
https://docs.microsoft.com/powershell/scripting/components/vscode/using-vscode => https://docs.microsoft.com/powershell/scripting/dev-cross-plat/vscode/using-vscode
PR Context
Fix #12775
PR Checklist

[x] PR has a meaningful title

Use the present tense and imperative mood when describing your changes

[x] Summarized changes
[x] Make sure all .h, .cpp, .cs, .ps1 and .psm1 files have the correct copyright header
[x] This PR is ready to merge and is not Work in Progress.

If the PR is work in progress, please add the prefix WIP: or [ WIP ] to the beginning of the title (the WIP bot will keep its status check at Pending while the prefix is present) and remove the prefix when the PR is ready.

Breaking changes

[x] None
OR
[ ] Experimental feature(s) needed

[ ] Experimental feature name(s):

User-facing changes

[x] Not Applicable
OR
[ ] Documentation needed

[ ] Issue filed:

Testing - New and feature

[x] N/A or can only be tested interactively
OR
[ ] Make sure you've added a new test if existing tests do not effectively test the code changed

Tooling

[x] I have considered the user experience from a tooling perspective and don't believe tooling will be impacted.
OR
[ ] I have considered the user experience from a tooling perspective and enumerated concerns in the summary. This may include:

Impact on PowerShell Editor Services which is used in the PowerShell extension for VSCode (which runs in a different PS Host).
Impact on Completions (both in the console and in editors) - one of PowerShell's most powerful features.
Impact on PSScriptAnalyzer (which provides linting & formatting in the editor extensions).
Impact on EditorSyntax (which provides syntax highlighting with in VSCode, GitHub, and many other editors).

Issue was fixed in MicrosoftDocs/PowerShell-Docs#6018 but let's bypass this redirect anyway.

@iSazonov can you approve?

@xtqqczze Thank you for your contribution!


Fixup regressions in sct_launcher & install_sct

This PR fixes:

#1899
#1897
#1895 (for good)

@zougloub do you want to keep the verbose=1 on Travis? I find it a bit difficult to navigate. I would suggest to do verbose=0 on Travis, and if there is a fail, we debug it locally.

@jcohenadad I didn't notice that the default param.verbose was 1, and didn't test on my computer at that time (it was quite late...). Yeah it shouldn't be verbose by default.


[e2e]: generic release main default slsa3

Repo: https://github.com/slsa-framework/example-package/tree/v24.0.475
Run: https://github.com/slsa-framework/example-package/actions/runs/8548250310
Workflow file: https://github.com/slsa-framework/example-package/tree/main/.github/workflows/e2e.generic.release.main.default.slsa3.yml
Workflow runs: https://github.com/slsa-framework/example-package/actions/workflows/e2e.generic.release.main.default.slsa3.yml
Trigger: release
Branch: v24.0.475
Date: Thu Apr  4 02:13:34 UTC 2024

Repo: https://github.com/slsa-framework/example-package/tree/v24.0.476
Run: https://github.com/slsa-framework/example-package/actions/runs/8563982507
Workflow file: https://github.com/slsa-framework/example-package/tree/main/.github/workflows/e2e.generic.release.main.default.slsa3.yml
Workflow runs: https://github.com/slsa-framework/example-package/actions/workflows/e2e.generic.release.main.default.slsa3.yml
Trigger: release
Branch: v24.0.476
Date: Fri Apr  5 02:14:03 UTC 2024
Tests are passing now. Closing this issue.


How to stop PositionStream after getting location updates.

💬 Questions and Help
How to stop PositionStream after getting location updates.
var geolocator = Geolocator();
         var locationOptions = LocationOptions(accuracy: LocationAccuracy.high, distanceFilter: 1, timeInterval: 1000);
         geolocator.getPositionStream(locationOptions).listen( (Position position) {
           if(position.accuracy > 10 ){
              // I want to cancel the position update
           }
 });

Do you actually manage to get the location? We've been creating issues for locations not retrieved.

Yes, am able to get the location updates. using
dependencies:
geolocator: '^3.0.0'

You should call the cancel method on the StreamSubscription instance that is returned when you called the listen method.
Something like this:
var geolocator = Geolocator();
var locationOptions = LocationOptions(accuracy: LocationAccuracy.high, distanceFilter: 1, timeInterval: 1000);
var subscription = geolocator.getPositionStream(locationOptions).listen( (Position position) {
           if(position.accuracy > 10 ){
              // I want to cancel the position update
              subscription.cancel();
           }
 });

You can also look at the example project (exact code is here: https://github.com/BaseflowIT/flutter-geolocator/blob/4601adc081c8df2831f676a86d8545a4680115a6/example/lib/pages/location_stream_widget.dart#L17)


refactor(form): improve form DX

This PR improves the overall DX of @hilla/form along with fixing some bugs.
The most noticeable changes are:

Improve TS support in all form components:

Reduce amount of as casts.
Reduce amount of any. According to TS recommendations, any was replaced preferably with unknown, in some cases with never.
tsc now can infer more types without the user support.

Models syntax is now more user-friendly.

Child types are created using a callback instead of a separate constructor + array of parameters.
Less magic in createEmptyValue method: no static inheritance, each class has its own method.
Due to these changes, the TS support became way better.

Updates for internal methods implementation to reduce cognitive load.

Multiple validation methods are now generators instead of arrays.
Now private, protected and public members are separated at the JS level:

Public members are just regular string methods.
Protected members are symbol methods.
Private members are new ECMA private members (with #).

For tests, the usage of Sinon spies is increased. They replace multiple manual implementations.

I apologize for the PR size. Unfortunately, it'd be hard to make small independent PRs for these changes since they are pulling one another.
There are going to be a couple of follow-on PRs that should be merged into this one to make all tests pass.

The first one will address the model generation. Since the structure has changed, it requires an update for the generator as well.
The second one will update react-form package to reflect changed made.

@sissbruecker, the problem is that there were several times I faced that issue during the development of react-form. It happens when you have the current implementation and try to use unknown instead of any everywhere (which is a TS recommended practice).
For example, the following code won't compile:
export function useFormPart<M extends AbstractModel<unknown>>(
  model: M,
): UseFormPartResult<ReturnType<M['valueOf']>, M> {
  const binderNode = getBinderNode(model);
  const field = useFields(binderNode);
  return {
    ...getFormPart(binderNode),
    field,
  };
}

It raises the following error:
TS2344: Type 'M' does not satisfy the constraint 'AbstractModel<ReturnType<M["valueOf"]>>'.
Type 'AbstractModel<unknown>' is not assignable to type 'AbstractModel<ReturnType<M["valueOf"]>>'.
The types returned by 'valueOf()' are incompatible between these types.
Type 'unknown' is not assignable to type 'ReturnType<M["valueOf"]>'.

So, you are forced to use any everywhere which reduces the type safety and could cause Eslint errors.
Then, about the same useFormPart hook. It has a different signature than useForm.
function useFormPart<M extends AbstractModel<any>>(model: M): UseFormPartResult<ReturnType<M['valueOf']>, M>;

useForm<T, M extends AbstractModel<T>>(
  Model: ModelConstructor<T, M>,
  config?: BinderConfiguration<T>,
): UseFormResult<T, M>;

When I was working on react-form, that part confused me. So I can imagine that it could easily confuse the developer who starts to work with the @hilla/form even though it could be an edge use case.
Third: this PR also reworks and simplifies the model implementation. Now you can create model by hands without additional cognitive load because everything is done via callbacks instead of constructors + array of their arguments.
Also, using only the specific model as an argument makes API much clearer and more responsive. And the internal cleanup increases the stability of the library.