Kosmos-2.5 - Image-to-markdown generation for images outside the sample-set provided is almost entirely garbled - output markdown is completely unusable.

Describe the bug
Model I am using: Kosmos-2.5
The problem arises when using:

[ x ] the official example scripts: Using the exact required custom-libraries and dependencies to run the supplied inference.py script. Same results obtained when run bare-metal or when extended via a simple Flask-API in a containerized environment: https://github.com/abgulati/kosmos-2_5-containerized

Description: Image-to-markdown generation for images outside the sample-set provided is almost entirely garbled - output markdown is completely unusable.
Elaborating in the examples below:
Example 1 - Using the sample in.png  example image provided with the model:

On running the inference.py script with --do_md for image-to-markdown generation:

Isolating the results:

Cleaning the results:

Perfect markdown output as rendered via https://markdownlivepreview.com/:

This confirms the model is working correctly!
Example 2 - Table from a Boeing manual:

Output of inference.py script with --do_md for image-to-markdown generation:

Copying, cleaning and generating a markdown preview of the results - completely garbled & unusable output:

Example 3 - Table of network connectors from my notes for the CompTIA Network+ exam:

Output of inference.py script with --do_md for image-to-markdown generation:

Copying, cleaning and generating a markdown preview of the results - completely garbled & unusable output:

Example 4 - Table of commons ports and services from my notes for the CompTIA Network+ & Security+ exams:

Output of inference.py script with --do_md for image-to-markdown generation:

Copying, cleaning and generating a markdown preview of the results - completely garbled & unusable output:

As demonstrated by these examples, markdown-generation for images outside the sample (training?) set is completely garbled and unusable. The first example establishes the model itself is working correctly.
Further, --do_ocr works perfectly and outputs high-accuracy, high-quality data.
To Reproduce
Steps to reproduce the behavior:

Run model for markdown generation: python3 inference.py --do_md --image_path/image.png -- ckpt ckpt.pt

Expected behavior
Respectably accurate markdown generation

Platform: WSL Ubuntu 22.04
Python version: v3.10.12
PyTorch version (GPU?): 2.5.0.dev20240705+cu124 for RTX 3090
Detailed system specs:

Intel Core i9 13900KF
Nvidia RTX 3090FE
32GB DDR5 5600MT/s (16x2)
Windows 11 - OS Build 22631.3737
CUDA 12.4

Flash-Attention-2 (v2.5.9.post1)
tiktoken 0.7.0
tqdm 4.66.4
omegaconf 2.0.6 (hydra-core 1.0.7)
boto3 1.34.140
iopath 0.1.10
fairscale 0.4.0
scipy 1.10.0
triton 2.3.1
https://github.com/facebookresearch/xformers.git@04de99bb28aa6de8d48fab3cdbbc9e3874c994b8
https://github.com/Dod-o/kosmos2.5_tools.git@fairseq
https://github.com/Dod-o/kosmos2.5_tools.git@infinibatch
https://github.com/Dod-o/kosmos2.5_tools.git@torchscale
https://github.com/Dod-o/kosmos2.5_tools.git@transformers

Following


run-converter.sh ERROR:gpu_init.cc(481)]

Running run-converter.sh from command prompt on Windows 10.
[main 2023-01-05T18:56:03.981Z] update#setState idle
[main 2023-01-05T18:56:06.074Z] UtilityProcess<1>: Creating new...
[main 2023-01-05T18:56:06.202Z] UtilityProcess<1>: received spawn event.
[main 2023-01-05T18:56:33.986Z] update#setState checking for updates
[main 2023-01-05T18:56:34.089Z] update#setState idle
[24828:0105/135802.331:ERROR:gpu_init.cc(481)] Passthrough is not supported, GL is disabled, ANGLE is
[main 2023-01-05T20:26:34.256Z] update#setState checking for updates
[main 2023-01-05T20:26:34.714Z] update#setState idle

The program is running for hours...

That doesn't look like any output from run-converter that I've ever seen. I suspect it's either an issue in the version of bash you're using, or your dotnet installation.
If you look at the shell script, you can run each individual command there and work out which part is hanging.

Closing as we haven't had any more feedback.


When using MirroredStrategy, the training process is stuck

System information

OS Platform and Distribution: Ubuntu 18.04
Python version: 3.6.9
CUDA/cuDNN version: 2x1080 Ti
Tensorflow version: 2.2.0

Describe the current behavior
I built a CRNN model for recognize the text, a CNN-RNN-CTC model, it work well without tf.distribute.Strategy, but when I follow the guide to move the creation and compiling of Keras model inside strategy.scope, It does not start distributed training like the guide. here is the output:
2020-03-13 06:19:04.839700: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1
2020-03-13 06:19:04.874256: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties:
pciBusID: 0000:01:00.0 name: GeForce GTX 1080 Ti computeCapability: 6.1
coreClock: 1.582GHz coreCount: 28 deviceMemorySize: 10.91GiB deviceMemoryBandwidth: 451.17GiB/s
2020-03-13 06:19:04.875077: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 1 with properties:
pciBusID: 0000:02:00.0 name: GeForce GTX 1080 Ti computeCapability: 6.1
coreClock: 1.582GHz coreCount: 28 deviceMemorySize: 10.92GiB deviceMemoryBandwidth: 451.17GiB/s
2020-03-13 06:19:04.875273: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1
2020-03-13 06:19:04.876811: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10
2020-03-13 06:19:04.878155: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10
2020-03-13 06:19:04.878383: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10
2020-03-13 06:19:04.879998: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10
2020-03-13 06:19:04.881014: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10
2020-03-13 06:19:04.884664: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7
2020-03-13 06:19:04.887439: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0, 1
2020-03-13 06:19:04.887722: I tensorflow/core/platform/cpu_feature_guard.cc:143] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
2020-03-13 06:19:04.892905: I tensorflow/core/platform/profile_utils/cpu_utils.cc:102] CPU Frequency:<PHONE_NUMBER> Hz
2020-03-13 06:19:04.893743: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x7f1224000b20 initialized for platform Host (this does not guarantee that XLA will be used). Devices:
2020-03-13 06:19:04.893784: I tensorflow/compiler/xla/service/service.cc:176]   StreamExecutor device (0): Host, Default Version
2020-03-13 06:19:05.147750: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x59b91e0 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices:
2020-03-13 06:19:05.147778: I tensorflow/compiler/xla/service/service.cc:176]   StreamExecutor device (0): GeForce GTX 1080 Ti, Compute Capability 6.1
2020-03-13 06:19:05.147788: I tensorflow/compiler/xla/service/service.cc:176]   StreamExecutor device (1): GeForce GTX 1080 Ti, Compute Capability 6.1
2020-03-13 06:19:05.150073: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties:
pciBusID: 0000:01:00.0 name: GeForce GTX 1080 Ti computeCapability: 6.1
coreClock: 1.582GHz coreCount: 28 deviceMemorySize: 10.91GiB deviceMemoryBandwidth: 451.17GiB/s
2020-03-13 06:19:05.150888: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 1 with properties:
pciBusID: 0000:02:00.0 name: GeForce GTX 1080 Ti computeCapability: 6.1
coreClock: 1.582GHz coreCount: 28 deviceMemorySize: 10.92GiB deviceMemoryBandwidth: 451.17GiB/s
2020-03-13 06:19:05.150926: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1
2020-03-13 06:19:05.150943: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10
2020-03-13 06:19:05.150958: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10
2020-03-13 06:19:05.150972: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10
2020-03-13 06:19:05.150986: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10
2020-03-13 06:19:05.151000: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10
2020-03-13 06:19:05.151015: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7
2020-03-13 06:19:05.154118: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0, 1
2020-03-13 06:19:05.154160: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1
2020-03-13 06:19:05.156174: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix:
2020-03-13 06:19:05.156195: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108]      0 1
2020-03-13 06:19:05.156206: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0:   N Y
2020-03-13 06:19:05.156214: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 1:   Y N
2020-03-13 06:19:05.158304: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 9569 MB memory) -> physical GPU (device: 0, name: GeForce GTX 1080 Ti, pci bus id: 0000:01:00.0, compute capability: 6.1)
2020-03-13 06:19:05.159295: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:1 with 10369 MB memory) -> physical GPU (device: 1, name: GeForce GTX 1080 Ti, pci bus id: 0000:02:00.0, compute capability: 6.1)
Epoch 1/5

The training process seems to be interrupted without any output, but the program runs out of graphics card memory and the program does not exit.
Describe the expected behavior
Start training just like without distributed strategy.
Standalone code to reproduce the issue
The full code can be view here

Could you try tf.experimental.output_all_intermediates(True)?
It would be also easier if you can provide a smaller reproduction.

The code is available in CRNN.tf2, To reproduce this error, clone it, add MirroredStrategy, then run it with example data by:
python train.py -ta example/annotation.txt -tf example -va example/annotation.txt -vf example -t example/table.txt

The error after add tf.compat.v1.experimental.output_all_intermediates(True):
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/op_def_library.py", line 470, in _apply_op_helper
    preferred_dtype=default_dtype)
  File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/ops.py", line 1317, in convert_to_tensor
    (dtype.name, value.dtype.name, value))
ValueError: Tensor conversion requested dtype float32 for Tensor with dtype resource: <tf.Tensor 'Identity/ReadVariableOp/resource:0' shape=() dtype=resource>

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "train.py", line 61, in <module>
    model = crnn(train_dl.num_classes)
  File "/workspace/model.py", line 152, in crnn
    x = original(img_input)
  File "/workspace/model.py", line 37, in original
    x = layers.BatchNormalization()(x)
  File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/engine/base_layer.py", line 921, in __call__
    outputs = call_fn(cast_inputs, *args, **kwargs)
  File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/layers/normalization.py", line 735, in call
    outputs = self._fused_batch_norm(inputs, training=training)
  File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/layers/normalization.py", line 598, in _fused_batch_norm
    _fused_batch_norm_inference)
  File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/utils/tf_utils.py", line 65, in smart_cond
    pred, true_fn=true_fn, false_fn=false_fn, name=name)
  File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/smart_cond.py", line 59, in smart_cond
    name=name)
  File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/util/deprecation.py", line 507, in new_func
    return func(*args, **kwargs)
  File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/ops/control_flow_ops.py", line 1177, in cond
    return cond_v2.cond_v2(pred, true_fn, false_fn, name)
  File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/ops/cond_v2.py", line 101, in cond_v2
    name=scope)
  File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/ops/cond_v2.py", line 235, in _build_cond
    true_intermediates = _get_intermediates(true_graph)
  File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/ops/cond_v2.py", line 443, in _get_intermediates
    if t in func_graph.inputs: continue
  File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/ops/math_ops.py", line 1491, in tensor_equals
    return gen_math_ops.equal(self, other, incompatible_shape_error=False)
  File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/ops/gen_math_ops.py", line 3224, in equal
    name=name)
  File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/op_def_library.py", line 506, in _apply_op_helper
    inferred_from[input_arg.type_attr]))
TypeError: Input 'y' of 'Equal' Op has type resource that does not match type float32 of argument 'x'.

Is batch normalization not available in distributed training?

For me, this issue is because I forgot to add drop_remainder=True in Dataset.batch function.

This problem still exists, I can't find a way to solve it. There are no errors, and the GPU is not calculating, but it takes up GPU memory.

@FLming Can you please try adding steps_per_epoch and validation_steps to model.fit and then test whether the error persists or not? Thanks!

I set steps_per_epoch to number of training samples // batch size and validation_steps to number of val samples // batch size, the problem still exists.

One more thing, although it’s a digression. Once I used DepthwiseConv2D to build the model, the same thing happened, there was no progress bar, no error, and the output was only
...
Epoch 1/x

By chance I switched the program from GPU to CPU, the error raised, that said current implementation only supports equal length strides in the row and column dimensions. Follow this tip, the program worked on GPU.
Will tensorflow lose some output prompts when running on the GPU?

@FLming Code that is running on CPU and GPU are similar but they are little different from implementation side so that the GPU accelerators are optimally used.
If this is resolved for you, then please close the issue. Thanks!

@jvishnuvardhan No, the problem still exists. The above is another problem that DepthwiseConv2D is't support different stride setting, and on GPU, the prompt doesn't appear.

Things have progressed. I removed custom Metric class in model.fit, the distributed training works normally, but why? Here is the definition
class WordAccuracy(keras.metrics.Metric):
    """
    Calculate the word accuracy between y_true and y_pred.
    """
    def __init__(self, name='word_accuracy', **kwargs):
        super().__init__(name=name, **kwargs)
        self.total = self.add_weight(name='total', dtype=tf.int32,
                                     initializer=tf.zeros_initializer())
        self.count = self.add_weight(name='count', dtype=tf.int32,
                                     initializer=tf.zeros_initializer())

    def update_state(self, y_true, y_pred, sample_weight=None):
        """
        Maybe have more fast implementation.
        """
        b = tf.shape(y_true)[0]
        max_width = tf.maximum(tf.shape(y_true)[1], tf.shape(y_pred)[1])
        logit_length = tf.fill([tf.shape(y_pred)[0]], tf.shape(y_pred)[1])
        decoded, _ = tf.nn.ctc_greedy_decoder(
            inputs=tf.transpose(y_pred, perm=[1, 0, 2]),
            sequence_length=logit_length)
        y_true = tf.sparse.reset_shape(y_true, [b, max_width])
        y_pred = tf.sparse.reset_shape(decoded[0], [b, max_width])
        y_true = tf.sparse.to_dense(y_true, default_value=-1)
        y_pred = tf.sparse.to_dense(y_pred, default_value=-1)
        y_true = tf.cast(y_true, tf.int32)
        y_pred = tf.cast(y_pred, tf.int32)
        values = tf.math.reduce_any(tf.math.not_equal(y_true, y_pred), axis=1)
        values = tf.cast(values, tf.int32)
        values = tf.reduce_sum(values)
        self.total.assign_add(b)
        self.count.assign_add(b - values)

    def result(self):
        return self.count / self.total

    def reset_states(self):
        self.count.assign(0)
        self.total.assign(0)

I found the reason, that is because I use dtype=tf.int32 in self.add_weight function.

I found the reason, that is because I use dtype=tf.int32 in self.add_weight function.
@FLming can you please share your experience how you found the root cause?

Comment and run, step by step, narrow the scope.

在 2020年10月8日，下午5:48，Boggis30<EMAIL_ADDRESS>写道：

I found the reason, that is because I use dtype=tf.int32 in self.add_weight function.
@FLming can you please share your experience how you found the root cause?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or unsubscribe.

Just a little sharing about this incident. Most of the problem take place by the loss and metric after running a step.
Make sure custom loss match this requirement for 2.x version:
If using tf.keras.losses classes (as in the example below), the loss reduction needs to be explicitly specified to be one of NONE or SUM. AUTO and SUM_OVER_BATCH_SIZE are disallowed when used with tf.distribute.Strategy. AUTO is disallowed because the user should explicitly think about what reduction they want to make sure it is correct in the distributed case. SUM_OVER_BATCH_SIZE is disallowed because currently it would only divide by per replica batch size, and leave the dividing by number of replicas to the user, which might be easy to miss. So instead we ask the user do the reduction themselves explicitly.

Just a little sharing about this incident. Most of the problem take place by the loss and metric after running a step.
Make sure custom loss match this requirement for 2.x version:
If using tf.keras.losses classes (as in the example below), the loss reduction needs to be explicitly specified to be one of NONE or SUM. AUTO and SUM_OVER_BATCH_SIZE are disallowed when used with tf.distribute.Strategy. AUTO is disallowed because the user should explicitly think about what reduction they want to make sure it is correct in the distributed case. SUM_OVER_BATCH_SIZE is disallowed because currently it would only divide by per replica batch size, and leave the dividing by number of replicas to the user, which might be easy to miss. So instead we ask the user do the reduction themselves explicitly.


making "/index.html" accessible from the browser

... to make "index.html" accessible from browser?
I'm referring to your basic sample project. Everything is working as long as I'm browsing
localhost:3000
But if I call
localhost:3000/index.html
I'm getting
404 - Oh no's! We couldn't find that page :(
That wouldn't be a big issue for me, but I'm hosting the app using sparkjava with a Jetty server behind. And this guy seems to rewrite any empty path to "index.html" -> 404 -> boom.
Any suggestion?

I'm having opened a stackoverflow thread on this. Help is really appreciated.
https://stackoverflow.com/questions/47904808/react-static-spark-java-jetty-proguard-all-involved-how-is-to-blame

Did you define a route in your static.config.js file for /index.html?

Hi @lottamus thanks for the response. Yes, I tried that, but it ends up in an error on npm run build
{ Error: ENOTDIR: not a directory, open '/Users/me/myproject/dist/index.html/index.html'
  errno: -20,
  code: 'ENOTDIR',
  syscall: 'open',
  path: '/Users/me/myproject/dist/index.html/index.html' }
npm ERR! code ELIFECYCLE

I don't think it could work that way.

Unfortunately, I don't think this issue is the responsibility of react-static. I think you'll have much better luck on your stack overflow question, seeing as this issue is directly related to your sparkjava + Jetty configuration.

@tannerlinsley Sorry, I disagree. Please refer to my initial post: To call a website with host:port/index.html is a pattern, which is as old as the Internet. This leads to 404, even in a basic react-static app from scratch.
And- what shall I say to the Jetty guys? The are embedded in Spark with such a limited amount of configuration possibilities. Even if I would have the hammer of a fully configurable Jetty instance, I wouldn't know the nail to hit. Then spark: They are just saying jetty," Here, serve public files from this and hat folder". OK, and it really works. If I put a plain "x.html" into the public folder, I can serve it, regardless of whether the JAR is obfuscated or not. I hear them saying: So what?
Back to the roots: react-static definitily serves index.html if it is called like localhost:5000, so why shoulnd't it be possible to do exactly the same, if one decorates this call with an index.html even though the file IS there..??

The server that comes with react-static is the serve package, which automatically does url rewriting, simulating the functionality of most static site hosting solutions. You can see the same functionality if you deploy your site to Netlify, AWS (with static-site bucket configurations), and even GoDaddy.
The files are built and available as index.html and so forth in your dist, which is the primary responsibility of React Static. React Static has no control over the hosting or serving of those files.

@tannerlinsley Well, that might be the case and I don't complain. I have another point.  My complain is the 404 even your server throws, if you go to localhost:5000/index.html. This is not something server related. It is something coming from the internal routing thing in react-static, because my index.html flashes up for 1/10 of a second and then the 404 appears. All what I want is:
If react-static is serving index.html if once opens localhost:5000 then I would like to have the same for localhost:5000/index.html

I see what you're saying. This is what's happening:

The page loads index.html just fine, and you see the simple HTML and CSS that ships on first load.
The js is parsed and React mounts. This is when you see the 404. This happens because the route path is seen as /index.html insteadof/. In fact, this exact same problem would happen if you were to use create-react-app and react-router. React router will look for a route component with path='/index.html'`. When that route is not matched in the component tree, the fallback route is used, which in React Static's case is the 404 component, naturally.

The only way I see around this is to somehow register your routes using the suffix, and this simply won't fly with the filesystem. It would attempt to create /index.html/index.html.
There are also a slew of other issue that could arise in react-router and react-static because of that change, so I wouldn't suggest attempting this.
In the end, you will be much better off by implementing a URL rewrite in your server configuration to remove the .html suffix and /index paths from routes that match an HTML file. This may or may not help you: http://www.eclipse.org/jetty/documentation/9.4.x/rewrite-handler.html.

Thanks for your post. This gave me an idea: I just set myself as 404 handler...

      {
        path: '/',
        component: 'src/containers/Home'
      },

      {
        is404: true,
//        component: 'src/containers/404',
          component: 'src/containers/Home'

Now my obufscated Jar works as well. The strange thing is, that I can localhost:5000 in with the non obfuscated version... The obfuscated version does that rewrite to index... As if some interface obtained by reflection cannot be seen anymore...

That's strange.  I honestly wish I knew more about your tech so I could help. But I'm pretty much a JS exclusive dev haha.  Keep this thread updated though so your findings can help posterity :)

Well, the tech is not too specific: It is a Java console app, which uses Spark (Spark is like Sinatra for Ruby) for doing a REST API.  Spark is using an internal Jetty server. Spark offers to host static pages (that's what I wanted to use). So I create my "static react app" and copy the whole DIST directory to a directory in the JAR, which is declared to be the public dir to Spark. From there the internal Jetty serves the DIST files.
Everything ok in development and production. But after proguarding the JAR, everything is still OK, but the static pages are now no longer working from ´localhost:5000(as before), but the Jetty server makes a rewrite to localhost:5000/index.html`, which has clashed with react-static.
The only theory I have is: I need to exclude a (still unknown) class from being obfuscated. Obfuscation and Reflection that does not really work together. Either spark or Jetty (I tend to believe it is Jetty) is deciding to rewrite to index.html. It didn't do it under unobfuscated condition...
So finding such a show stopper is not that easy, from my experiences.

For anyone stumbling upon this issue, I was able to fix it using JS creating a script tag in the head with the following code
window.location.pathname === '/index.html' && window.history.replaceState('', '', '/');

This method doesnt even redirect and will not show the 404 page because the /index.html path wont be there when react loads.
Maybe something similar can be implemented in the code base of react-static or maybe just documented so that people is aware of this.
In my case it was important because google for some reason was indexing /index.html and all my clients where getting to the 404 page.

this probably is a simple fix we can do inside react-static somehow. i agree it is annoying, although i have generally always addressed it from the CDN side (i use netlify)

I have my site behing nginx and after some hours of trying different server configs I had no luck so I fixed it this way.
Havent seen any side effects yet. (I dont think it can cause an issue somewhere else, but not sure.)

I still think that this should not be addressed by react-static. However, it's something I'll see if we can fix during a refactor soon. Closing for now.

In my use case we've tried to publish a site generated with react-static as a job artifacts in Gitlab.
In GL the artifacts can be rendered in GL pages. However, for job artifacts the index.html is visible in the browser URL. This causes the page to construct invalid paths when loading resources.
Is there any work around for such issue?


ES: 1203.json COSE seems to be invalid

https://raw.githubusercontent.com/eu-digital-green-certificates/dgc-testdata/main/ES/2DCode/raw/1203.json
When decoding the COSE input of this file, the payload decodes as a BigInt, not a byte array. Also in cbor.me it seems the encoding is different from "normal" files.
Anyone else having issue with this file?

Hi we have check that test, and has previous errors that We have solved. I think we have a problem with the pull request. We remove that test and we are checking what happen with that test


split array and move first part to end

Hi @ts4475, I request you to Raise the pull request to the sub branch in the repository


Getting Cloudflare challenge screen when calling https://www.nctx.co.uk/stops/3390SH15

In Postman or Thunder Client (vs code extension) running a GET for https://www.nctx.co.uk/stops/3390SH15
shows a challenge question
I saw it when I tried the first time via browser
However, my worker seems to be working at the moment:
https://cloudflare-nctx-bus-stop-api.coding-to-music.workers.dev/?stopId=3390FO07
Cool project btw, my repo is here
https://github.com/coding-to-music/cloudflare-nctx-bus-stop-api
When they give 503 with the Cloudflare challenge then my worker returns null for the stops

Hi there - thanks for getting in touch, glad you found the project useful, do you have plans to build a front end or something?
Thanks for letting me know about the 503, that may explain why I sometimes see null responses, and why people outside of the UK have reported this too.  As I am effectively screen scraping the bus company's site, I guess this sort of thing is to be expected periodically.  I'll have a play with things like specifying a user agent when I fetch the source for their page and see if that makes it look less suspect to Cloudflare.

I read about your tool on the Cloudflare blog, funny that now the bus company is using a different Cloudflare tool to "protect" their site, but it's more hobbyist use, and really they should not block every bot attempt, just block excessive attempts.

I'm copying this Gatsby site today, it's one of the best sites I've seen and looks like it could be modified for many uses, such as the bus api
https://github.com/coding-to-music/spacexstats-gatsby-chartjs
You might reach out to the bus company and tell them your scraper got some good press and please stop blocking innocent uses...

Thanks - I will definitely reach out to them but will try a few tactics to look like a normal browser first - I didn't know I'd made the Cloudflare blog, do you happen to have a link as I am struggling to find one?

The SpaceX site looks great BTW.

They called it train departures, at the bottom https://blog.cloudflare.com/platform-week-2022-wrap-up/

This one is cool, I think API's and feeds are recommended rather than scraping because the page inevitably changes
https://github.com/coding-to-music/stm-gtfs-realtime


Including info on OpenID Additional Parameters

Description
Including info on newly added "Additional Parameters" support for Open ID
Issues Resolved
Fixes https://github.com/opensearch-project/documentation-website/issues/5598
Checklist

[x] By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license and subject to the Developers Certificate of Origin.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

@Naarcha-AWS The code change corresponding to this documentation has not been merged. Can we revert this change in 2.11 and include it in the next version release?


support quotas

The more I've thought about this approach dataset + nfs share the more I like it.  I'd really like to start exploring supporting setting a quota on the dataset when created but am having troubles getting the build to work.  Being new to go makes this a bit of a stretch, but I'd like to help.  If you've got some pointers on building and running this from a development perspective that would be great.

[signal SIGSEGV: segmentation violation code=1 addr=32 pc=94347298903359]

goroutine 1 [running]:
panic
	/build/gcc/src/gcc/libgo/go/runtime/panic.go:557
path.StripVendor..func1
	/build/glide/src/build/src/github.com/Masterminds/glide/path/strip.go:28
filepath.walk
	/build/gcc/src/gcc/libgo/go/path/filepath/path.go:377
filepath.walk
	/build/gcc/src/gcc/libgo/go/path/filepath/path.go:381
filepath.walk
	/build/gcc/src/gcc/libgo/go/path/filepath/path.go:381
filepath.walk
	/build/gcc/src/gcc/libgo/go/path/filepath/path.go:381
filepath.walk
	/build/gcc/src/gcc/libgo/go/path/filepath/path.go:381
path_filepath.Walk
	/build/gcc/src/gcc/libgo/go/path/filepath/path.go:403
github_com_Masterminds_glide_path.StripVendor
	/build/glide/src/build/src/github.com/Masterminds/glide/path/strip.go:22
github_com_Masterminds_glide_action.Install
	/build/glide/src/build/src/github.com/Masterminds/glide/action/install.go:62
main.commands..func12
	/build/glide/src/build/src/github.com/Masterminds/glide/glide.go:510
github_com_Masterminds_glide_vendor_github_com_codegangsta_cli.HandleAction
	/build/glide/src/build/src/github.com/Masterminds/glide/vendor/github.com/codegangsta/cli/app.go:490
github_com_Masterminds_glide_vendor_github_com_codegangsta_cli.Command.Run
	/build/glide/src/build/src/github.com/Masterminds/glide/vendor/github.com/codegangsta/cli/command.go:210
github_com_Masterminds_glide_vendor_github_com_codegangsta_cli.App.Run
	/build/glide/src/build/src/github.com/Masterminds/glide/vendor/github.com/codegangsta/cli/app.go:255
main.main
	/build/glide/src/build/src/github.com/Masterminds/glide/glide.go:104
make: *** [Makefile:27: vendor] Error 2

Nice !
Do you succeeded in compiling ? It seems that it's an error when running, isn't it ?
Also, the code currently support only in-cluster configuration so you will need a Kubernetes cluster to run it (minikube should do it).

Ok, I built it successfully using glide 0.13.0 and go 1.10 using:
make clean
make vendor
make

Don't use glide up command as previously stated, it's messing up the vendor directory.

@nmaupu yeah, that's what I've been doing.  On arch linux here.
go version
go version go1.10.3 linux/amd64

glide --version
glide version 0.13.0-dev

What's your setup.  I'm sure I can sandbox it in some way if necessary.

You could try some go building docker image.
Also, I know that go is very sensible about changing path where it has been configured for (check GOPATH and GOROOT env var).

@nmaupu yeah, I've been using a docker image for development just to build.  I'm sure I'll get to the bottom of it eventually on my host machine.

Closed by PR #6


nixos/acme: Add option to make certificates world readable

Motivation for this change
I generate wildcard certificates using a local BIND instance and use them for a lot of different services, and even in containers by mounting them. It is quite a pain to figure out the users of all the services I'm running so I can add them to the acme group. I'm willing to accept the security risk of making the certificates world readable (if someone manages to break into my system, I have bigger problems than comprimised SSL keys).
Things done

[x] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
Built on platform(s)

[x] NixOS
[ ] macOS
[ ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
[ ] Tested execution of all binary files (usually in ./result/bin/)
[ ] Determined the impact on package closure size (by running nix path-info -S before and after)
[x] Ensured that relevant documentation is up to date
[x] Fits CONTRIBUTING.md.

Why are you not using a reverse proxy that does TLS? If you are using nginx then you only need this line and you are done: users.users.nginx.extraGroups = [ "acme" ];

I am using nginx as a reverse proxy, it's just that I have more than a web server running. I don't think you can reverse proxy IMAPS/SMTPS with nginx.

Nobody should use this. We should encourage service module authors to add options for acme instead. That way, you won't have to deal with configuring file paths either.
Adding a service user to a group isn't a hard problem, so we shouldn't accept this as a workaround.

 I've added all the service users to the group and it isn't very hard to do.


token data text areas are black on black in v0.4.0

tested on Windows, don't know how it is otherwise

sadtrombone.wav
Will look into it


Set outbound interface, like ping -I option

Hello,
I attempted to implement a useful feature - sending data packets from a specific network interface, similar to what the "ping" command's "-I" option does in Linux.
Hoping I did it correctly. Test successfully passed on Linux OS.
Thanks!
The issues related to this feature:
go-ping/ping#111
go-ping/ping#204

This needs a DCO sign-off. You can use git commit -s --amend to add it.

FWIW, I would like to see this feature added in order to use VRF with this library.

I need this feature really bad. Can anyone merge this?

The PR needs to be rebased if it is to have any chance of getting merged.

Checking on this PR?  I could use it.  Is it dead?

Hi, any updates on this PR? It is quite useful when debugging the network on hosts with multiple net interfaces.

I need this to use link-local IPv6 addresses

At least the PR has conflicts, @ilolicon can you resolve them?

At least the PR has conflicts, @ilolicon can you resolve them?

Sorry, I was too busy recently and didn't pay attention to the news. It seems that @MatthieuCoder has resolved the conflict.


[Request] Automatic Routing once you reach Orbit

Hey there,
I was wondering if it would be possible for you to add a configuration that would automatically route to the next moon once you reach orbit after leaving the Company facility and other moons? Other mods have this functionality, HOWEVER, they implement it in a bad way - they randomly choose a moon once you pull the lever which I've noticed that this breaks modded moons. However, I believe that if you randomly route before pulling the lever (aka, when the ship reaches orbit), in theory, it should work just fine.
Maybe someone sort of configuration option to stop people from manually selecting moons and spamming the route random command might be cool too? :D
Hopefully I explained my ideas well...if you have any questions about what I'm talking about, let me know! :) If you have any way I can donate to you that'd be very cool too - cheers!

While this is a great idea I think it would be better suited for another mod. The point of this mod is having the option to ignore the randomly chosen moon, it would be too much effort to remove that option entirely just for the sake of a config that's turned off by default and 90% of players won't even know exists
Closing as not planned for this mod


add: dnf5-default-package-manager

tested; confirm working.
merging into tree

actually, merging into tree as optional package. can be toggled using beta flags anyway


reset raft term to self.term in initial, not 0

When creating a network in test, the raft's term should not be set to 0 if it has some initial state.
Corresponding code in etcd

This is part of pr etcd/pull#6624, we should port this pr first.

@BusyJay , Yep, this has been implemented in #1330 , PTAL


Problem with the point animation

the point kinda shakes while moving between the tabs as this gif shows:

Maybe you will show your adapter?


LUIS and QnA Maker deprecated/retired, replaced by CLU and Custom QA

Is your feature request related to a problem? Please describe.
As of 1st April, you cannot create new LUIS services anymore, it has been replaced by Conversational Language Understanding (CLU), see https://www.luis.ai/

Also QnA Maker is deprecated and has been replaced by Custom Question Answering, see https://www.qnamaker.ai/

Therefore, the Python code samples provided here don't work anymore.
Describe the solution you'd like
We need updated Python code samples reflecting the transition from LUIS to CLU and from QnA Maker to Custom QA. This includes any necessary changes to the import statements, class instantiation, and method calls to ensure compatibility with the new services.
Describe alternatives you've considered
One alternative is to manually modify the existing code samples to align with the changes in services. However, this approach is time-consuming and error-prone, especially for developers unfamiliar with the intricacies of the transition. Therefore, having officially updated Python code samples provided by Azure would greatly expedite the process for developers migrating to CLU and Custom QA.

@d-kleine thanks for your issue. We're currently working on cleaning up deprecated libraries and documentation so that it's more clear which client libraries to use. Our libraries that target LUIS and QnAMaker will soon be deprecated:
LUIS: azure-cognitiveservices-language-luis
QnAMaker: azure-cognitiveservices-knowledge-qnamaker
The libraries you should use for CLU and QA include the following:
CLU: azure-ai-language-conversations
QA: azure-ai-language-questionanswering

Therefore, the Python code samples provided here don't work anymore.

For CLU, there is a sample that shows how to use the old LUIS service (sample_analyze_orchestration_app_luis_response.py) and we will remove it. The other samples should work with the current CLU service. Is there something you found that didn't work?

Thanks! I was just asking to update these Azure SDK for Python samples to be working with CLU and Custom QA, along with an updated documentation and code comments.

CLU: https://github.com/Azure/azure-sdk-for-python/blob/main/sdk/cognitivelanguage/azure-ai-language-conversations/samples/sample_analyze_orchestration_app_luis_response.py
Custom QA: https://github.com/Azure/azure-sdk-for-python/blob/main/sdk/cognitivelanguage/azure-ai-language-conversations/samples/sample_analyze_orchestration_app_qna_response.py

I haven't tested it yet because the Python samples looked deprecated. I wanted to try to set up an QA Bot with Azure Bot with Python (as there are no Python samples in the BotBuilder Samples provided for that).

@d-kleine from my understanding you want an example of using CLU and Custom QA on an orchestration project. The samples you need are

Custom QA: https://github.com/Azure/azure-sdk-for-python/blob/main/sdk/cognitivelanguage/azure-ai-language-conversations/samples/sample_analyze_orchestration_app_qna_response.py
CLU: https://github.com/Azure/azure-sdk-for-python/blob/main/sdk/cognitivelanguage/azure-ai-language-conversations/samples/sample_analyze_orchestration_app_conv_response.py

The only sample that will be deprecated is the

LUIS: https://github.com/Azure/azure-sdk-for-python/blob/main/sdk/cognitivelanguage/azure-ai-language-conversations/samples/sample_analyze_orchestration_app_luis_response.py

But click here for more information on Orchestration workflow

Hi @d-kleine - Thanks for your patience! We're in the process of removing the deprecated sample_analyze_orchestration_app_luis_response.py sample to remove any confusion there.
We've also officially deprecated these packages:

https://pypi.org/project/azure-cognitiveservices-language-luis/
https://pypi.org/project/azure-cognitiveservices-knowledge-qnamaker/


Share uploaded file to public

Overview
I'm an Android developer and sometime need to upload my apk to google drive, but it's time consuming to open google drive and share it manually. Before i use windows and install ShareX
just right click -> upload. The file will uploaded and when it finished it will copy the link to the clipboard. But in mac i didn't found app like thta
Suggestion
It's be better if in future those feature will be added

Hello @aprakoso98
Thanks for opening this issue!
It is a very nice suggestion :smile:

Hello @aprakoso98,
I have developed a command on 1.2.0 to copy file or folder URL to clipboard, through Drive view.
Although this is not exactly what you asked for, with this command we can easily paste the URL on web browser and share file to public through Google Drive itself.

It was easier to develop it this way and I have been missing the feature to copy URL to clipboard, for other reasons.
Hope it helps! :smile:
Again, thanks for opening this issue and thanks for using the extension.
Feel free to open new issues as you need.
Regards,
Gustavo Cassel

Hi @GustavoASC, I love using this extension.
And thanks for developing my feature request this is much enaugh for me 🙏

Hi @aprakoso98
In case you still use the extension, could you rate it on Marketplace please? :smile:
Thank you very much!

Hi @GustavoASC,
I'm installed Code - Insider, and got error like this

Hi @aprakoso98
Thanks for your comment :smile:
This problem usually happens when one configures an incorrect credentials.json file, as explained on issue number #7.
The credentials is probably not configured as a Desktop app.
On the issue mentioned above, a proper warning notification will be shown.
Unfortunately I haven't had the time to add a check yet...


fix: Exposing too many variables in the example

Description
Instead of exposing schema and topic variables in the examples ,hardcoded the data in main.tf of examples.Issue
Release required?

[ ] No release
[x] Patch release (x.x.X)
[ ] Minor release (x.X.x)
[ ] Major release (X.x.x)

Release notes content
Instead of exposing schema and topic variables in the examples ,hardcoded the data in main.tf of examples.
Run the pipeline
If the CI pipeline doesn't run when you create the PR, the PR requires a user with GitHub collaborators access to run the pipeline.
Run the CI pipeline when the PR is ready for review and you expect tests to pass. Add a comment to the PR with the following text:
/run pipeline

Checklist for reviewers

[ ] If relevant, a test for the change is included or updated with this PR.
[ ] If relevant, documentation for the change is included or updated with this PR.

For mergers

Use a conventional commit message to set the release level. Follow the guidelines.
Include information that users need to know about the PR in the commit message. The commit message becomes part of the GitHub release notes.
Use the Squash and merge option.

/run pipeline

cc: @SarikaSinha

@aatreyee2506- pre-commit failed, please check

@aatreyee2506, Pre-commit failed again, please always run pre-commit in your local before you push the code.

/run pipeline

/run pipeline

/run pipeline

:tada: This PR is included in version 3.0.2 :tada:
The release is available on:

GitHub release
v3.0.2

Your semantic-release bot :package::rocket:


Improve English translations

This includes a few small improvements to the English version.

Thank you :)

Thank you :)
Can you update the files in the release?

Thank you :)
Can you update the files in the release?

I released files as v1.25-eng here.
https://github.com/Quandra/SMHatchingRNGTool/releases/tag/v1.25-eng

ありがとうございます！


Proposal: YurtCluster Operator for converting K8S to OpenYurt

Signed-off-by: nunu<EMAIL_ADDRESS>
What type of PR is this?

Uncomment only one  /kind <> line, hit enter to put that in a new line, and remove leading whitespace from that line:
/kind bug
/kind documentation
/kind enhancement
/kind good-first-issue
/kind feature
/kind question
/kind design
/sig ai
/sig iot
/sig network
/sig storage
/sig storage

What this PR does / why we need it:
Which issue(s) this PR fixes:

Fixes #
Special notes for your reviewer:

Does this PR introduce a user-facing change?

other Note

/assign @fei-guo

/kind feature

create this new pull request for clean merge operation. sorry for any extra efforts and confusion caused!


reduce split overflow

If a word is too large for a line, force a split just after it.
I removed many spaces from the first line of example/document to have a word larger than the paragraph size.
It is expected, that the word is larger, but it is unexpected that the rest of the sentence is not sent to a new line.
Before this MR:

After this MR:

In the end this was a bug in the split functionality, thanks for the effort though!


feat: add support for jump the queue label

:pencil: Description

Adds functionality to allow "jumping" the merge queue in the manage-merge-queue helper. This means if the JUMP THE QUEUE label is added to a PR in the queue, it will be automatically moved to #1 in the queue and everything else will be bumped down by 1.
Also refactors the code and tests a bit.

:tada: This PR is included in version 1.6.0 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


chore(release): v0.5.0 🎉

:robot: I have created a release beep boop
0.5.0 (2023-07-26)
Features

events: expose method to create exampe s3 event (c7d3dd6)

Bug Fixes

assumptions: remove sanity check of stage vs env variable (c35dc28)
practs: upgrade best practices (9365aed)

This PR was generated with Release Please. See documentation.

:robot: Release is at https://github.com/ehmpathy/simple-lambda-testing-methods/releases/tag/v0.5.0 :sunflower:


test: 🥳 close some recently introduced test gaps

PR Checklist
Please check if your PR fulfills the following requirements:

[x] The commit message follows our guidelines: https://github.com/ngneat/falso/blob/master/CONTRIBUTING.md#commit
[x] Tests for the changes have been added (for bug fixes / features)
[x] Docs have been added / updated (for bug fixes / features)

PR Type
What kind of change does this PR introduce?

[ ] Bugfix
[ ] Feature
[ ] Code style update (formatting, local variables)
[x] Refactoring (no functional changes, no api changes)
[ ] Build related changes
[ ] CI related changes
[ ] Documentation content changes
[ ] Other... Please describe:

What is the current behavior?

Issue Number: N/A
What is the new behavior?
Does this PR introduce a breaking change?
[ ] Yes
[x] No

Other information

Please check the return type and not that it's just toBeTruthy().


New ITO_flow, add payment options

I've copied the buy_tokens_ITO_v2 flow and made v3. It is identical except for offering extra payment options,
EUR, GBP, UAED
Corresponding flow choice variables are
EUR_cur
GBP_cur
AED_cur
Same bank as before, but the account numbers are
AED: AE500330000019120028155
USD : AE230330000019120028156
Euro : AE930330000019120028157
GBP: AE780330000019120032792
We will revise the iTFT token purchase agreement ASAP and link it here.

We also still need to decide on conversion rates.

We can use the hourly conversion rates and mention those in the contract. The contract is being checked by Pierre.

Contract v1.5 is finalised at
https://docs.google.com/document/d/162H9QDswivRNwDhxIDmvodzXNsrNtmR9fuGZKshEE_w/edit#
We will explicitly mention the conversion rates to be used. This implies that the amount of tokens is known at signing time for FIAT purchases, but (still) not for BTC.

https://github.com/threefoldfoundation/app_backend/commit/24e71d3f2e54cfcd8005b53c130bb55b89dfccb5


[RFC] Create connection and authenticate with a single function

Current State
The current API forces users to do some state handling themselves during connection setup. Also throwing functions are mixed with functions that return EventLoopFutures. From the README:
import PostgresNIO

let eventLoop: EventLoop = ...
let conn = try PostgresConnection.connect(
    to: .makeAddressResolvingHost("my.psql.server", port: 5432),
    on: eventLoop
).wait()
defer { try! conn.close().wait() }

There is a couple of issues with this:

.makeAddressResolvingHost("my.psql.server", port: 5432) is a throwing function
PostgresConnection.connect returns an EventLoopFuture.
The connection is only setup and the user needs to authenticate from here on. In an error case the user needs to teardown the connection themselves.

If users want to create a connection themselves they are forced to write this code, which has a lot of implicit connection state:
eventLoop.submit { () throws -> SocketAddress in
    // callback only to transform the throwing SocketAddress into a Future for consistency
    try .makeAddressResolvingHost("my.psql.server", port: 5432)
}.flatMap {
    // connect to the server. Can setup TLS
    PostgresConnection.connect(to: $0, on: eventLoop)
}.flatMap { connection in
    // authenticate to database
    connection.authenticate(
        username: env("POSTGRES_USER") ?? "vapor_username",
        database: env("POSTGRES_DB") ?? "vapor_database",
        password: env("POSTGRES_PASSWORD") ?? "vapor_password"
    ).map {
        // map to connection to return the connection to the user
        return conn
    }.flatMapError { error in
        // close connection in case of auth error
        conn.close().flatMapThrowing {
            // rethrow error for the user to capture
            throw error
        }
    }
}

Changes
With the StateMachine PR #135 a new underlying PSQLConnection has landed that supports connection creation and authentication with a single call. This PR brings this much easier API to the public interface:
Additions
class PostgresConnection {
    /// Create a new Postgres Connection to a Postgres Server.
    ///
    /// - Parameters:
    ///   - hostname: The server's hostname as domain or IP address.
    ///   - port: The server's port. Defaults to the Postgres default port 5432.
    ///   - username: The username to authenticate with to the remote server.
    ///   - password: An optional password to authenticate with to the remote server.
    ///   - database: The database name to connect to
    ///   - tlsConfiguration: An optional `TLSConfiguration` that configures the TLS connection to the server. If
    ///                       none is provided the connection to the server is created without TLS.
    ///   - logger: A logger to use for connection state changes and action. The logger is only with the `debug` and
    ///             `trace` log level.
    ///   - eventLoop: The SwiftNIO `EventLoop` the connection should be created on.
    /// - Returns: An `EventLoopFuture` which provides the `PSQLConnection` if the connection creation and
    ///            authentification was successful. Otherwise an `Error`.
    public static func connect(
        hostname: String,
        port: Int = 5432,
        username: String,
        password: String?,
        database: String?,
        tlsConfiguration: TLSConfiguration?,
        logger: Logger = Logger(label: "codes.vapor.postgres", factory: { _ in SwiftLogNoOpLogHandler() }),
        on eventLoop: EventLoop
    ) -> EventLoopFuture<PostgresConnection>
}

For users that want to connect to their Postgres server via unix sockets, we offer a new advanced API:
    /// Create a new Postgres Connection to a Postgres Server.
    ///
    /// - Parameters:
    ///   - address: The remote server's address as a `SocketAddress`.
    ///   - serverName: The remote server's name to use for domain name certificate verification, if a
    ///                 `TLSConfiguration` is supplied.
    ///   - username: The username to authenticate with to the remote server.
    ///   - password: An optional password to authenticate with to the remote server.
    ///   - database: The database name to connect to
    ///   - tlsConfiguration: An optional `TLSConfiguration` that configures the TLS connection to the server. If
    ///                       none is provided the connection to the server is created without TLS.
    ///   - logger: A logger to use for connection state changes and action. The logger is only with the `debug` and
    ///             `trace` log level.
    ///   - eventLoop: The SwiftNIO `EventLoop` the connection should be created on.
    /// - Returns: An `EventLoopFuture` which provides the `PSQLConnection` if the connection creation and
    ///            authentification was successful. Otherwise an `Error`.
    public static func connect(
        address: SocketAddress,
        serverName: String? = nil,
        username: String,
        password: String?,
        database: String?,
        tlsConfiguration: TLSConfiguration?,
        logger: Logger = Logger(label: "codes.vapor.postgres", factory: { _ in SwiftLogNoOpLogHandler() }),
        on eventLoop: EventLoop
    ) -> EventLoopFuture<PostgresConnection>

Deprecations
The current connect and authenticate methods are deprecated.
extension PostgresConnection {

    @available(*, deprecated, message: "Use the new `PostgresConnection.connect(hostname:port:username:password:database:tlsConfiguration:logger:on eventLoop:)` that allows you to specify username, password and database directly when creating the connection.")
    public static func connect(
        to socketAddress: SocketAddress,
        tlsConfiguration: TLSConfiguration? = nil,
        serverHostname: String? = nil,
        logger: Logger = .init(label: "codes.vapor.postgres"),
        on eventLoop: EventLoop
    ) -> EventLoopFuture<PostgresConnection>

    @available(*, deprecated, message: "Use the new `PostgresConnection.connect(hostname:port:username:password:database:tlsConfiguration:logger:on eventLoop:)` that allows you to specify username, password and database directly when creating the connection.")
    public func authenticate(
        username: String,
        database: String? = nil,
        password: String? = nil,
        logger: Logger = .init(label: "codes.vapor.postgres")
    ) -> EventLoopFuture<Void>
}

Alternatives
Instead of using eight parameters for the connect function, we could use the a Config struct like this from PSQLConnection: https://github.com/vapor/postgres-nio/blob/main/Sources/PostgresNIO/New/PSQLConnection.swift#L12-L77

Nice, this certainly simplifies things a lot.
The only concern I have with this is that it gives username/password a privileged place in the API but PostgreSQL supports many auth mechanisms.
I think before committing to new APIs, we should think through how we could in the future evolve the APIs to support the other mechanisms. What doesn't work super well IMHO is to have a large number of connect methods, each only differing in the auth mechanism. (just for the record, I'm not saying that we should implement all this now but we should have a clear API evolution plan on how we could add things in the future).
One particular auth mechanism to consider is cert auth because it has implications on the TLS configuration too. So we can't fully separate TLS & auth for all the mechanisms. I'm sure @lukasa has opinions here, we've learnt how not to do a few things from AHC :).

Yeah the mere presence of cert auth interacts weirdly with all of this.
I'm with @weissi: the plan should be to find a way to encode auth fairly generally, even if we don't actually do it right away.

Thanks, for your feedback @weissi.
We will always need a username and potentially a database. The auth mechanisms
Let's look at all Authentication methods one by one. From Postgres docs 20.3 Authentication Methods

Trust authentication, which simply trusts that users are who they say they are.

Even though we don't need to authenticate we need to supply a username during startup.

Password authentication, which requires that users send a password.

We need a username and a password. The password may be send in one of three modes:

scram-sha-256
md5
password

All are supported by the current implementation \o/. Thanks @gwynne for providing scram-sha-256.

GSSAPI authentication, which relies on a GSSAPI-compatible security library. Typically this is used to access an authentication server such as a Kerberos or Microsoft Active Directory server.

Currently not supported. Quick look... I have no idea what is going on there.

SSPI authentication, which uses a Windows-specific protocol similar to GSSAPI.

Currently not supported. Quick look... I have no idea what is going on there.

Ident authentication, which relies on an “Identification Protocol” (RFC 1413) service on the client's machine. (On local Unix-socket connections, this is treated as peer authentication.)

Currently not supported. IIUC this doesn't need any user input.

Peer authentication, which relies on operating system facilities to identify the process at the other end of a local connection. This is not supported for remote connections.

This should work. No user input needed. Looks to the client like a trusted connection.

LDAP authentication, which relies on an LDAP authentication server.

Similar to the password dance. The actual authentication is just delegated to a LDAP server.

RADIUS authentication, which relies on a RADIUS authentication server.

Similar to the password dance. The actual authentication is just delegated to a RADIUS server.

Certificate authentication, which requires an SSL connection and authenticates users by checking the SSL certificate they send.

I assume this is from a protocol standpoint the same as trust authentication. Certificate cn name must match the username.

PAM authentication, which relies on a PAM (Pluggable Authentication Modules) library.

Similar to the password dance. The actual authentication is just delegated to a PAM service.

BSD authentication, which relies on the BSD Authentication framework (currently available only on OpenBSD).

Similar to the password dance. The actual authentication is just delegated to a BSD Authentication service.

The startup message requires a username and an optional database name (the fallback is the username).
On the one hand from a first glance nearly all auth methods need either nothing or only a password to work. Exception from what I can see right now is GSSAPI/SSPI.
On the other hand we have quite a bulk of additional Parameter Key Words in the libpq docu: https://www.postgresql.org/docs/13/libpq-connect.html#LIBPQ-PARAMKEYWORDS

To allow for different authentication modes down the road and given that we don't know what authentication scheme is, that the server requires we will need to create some kind of AuthenticationHandler with callbacks for different auth methods.
struct AuthenticationHandler: ExpressibleByStringLiteral {
  var providePassword: ((EventLoop) -> EventLoopFuture<String>)?

  static func password(_ pw: String) -> Self {
    init(providePassword: {
      $0.makeSucceededFuture(pw)
    })
  }

  init(stringLiteral value: String) {
    self.providePassword: {
      $0.makeSucceededFuture(value)
    })
  }
}

class PostgresConnection {
    public static func connect(
        hostname: String,
        port: Int = 5432,
        username: String,
        database: String?,
        authentication: AuthenticationHandler?,
        tlsConfiguration: TLSConfiguration?,
        logger: Logger = Logger(label: "codes.vapor.postgres", factory: { _ in SwiftLogNoOpLogHandler() }),
        on eventLoop: EventLoop
    ) -> EventLoopFuture<PostgresConnection>
}


[BUG] when i open it, it js closes gng

Acknowledgement of preliminary instructions

[X] I have read the preliminary instructions, and I am certain that my problem has not already been addressed.

What problem did you encounter?
when i open it, it js closes gng without even opening roblox or bloxstrap. like i cant even boot that ho

If your problem is with Roblox itself, check to see if it happens without Bloxstrap. Please only open an issue if your problem only happens with Bloxstrap, as anything else is out of my control.

Issue has been awaiting clarification for over a month, closing as stale. If you have anything further to add, just respond and I'll reopen.


Documentation system

Right now there are some docs in the wiki. We are starting to create more docs (see #102) so now is the time to decide what we should do.
I am not a big fan of restructured text but still vote for sphinx. Preferably hosted on/integrated with everware/everware.github.io. Travis could rebuild and push the docs when it builds master.

There is also #6. Need to check what of it still applies/is outdated.

/me had some positive experience with sphinx (https://github.com/yandex/rep/tree/master/docs, https://github.com/openml/openml-python/blob/develop/doc/Makefile) so let's borrow from those )

sphinx generates a Makefile for you right? So from the top level all we need is make doc that defers to the Makefile generated by sphinx.
Add building the docs to build_tools/ as a "test" so we are sure they build.
On merging a PR need to rebuild the docs and upload them (unless they haven't changed). This is implemented in https://github.com/scikit-learn/scikit-learn/tree/master/build_tools/circle should check how much can be reused for travis.

Looks like the openml Makefile is indeed the one generated by sphinx. Nice.


cast does not accept --name option

$ cast --version
cast version 0.0.5
On the examples presented on the main page, global option "--name" is used however version 0.0.5 downloaded from latest does not accept the option.  Output returned:
$ cast --name patio quit
Incorrect Usage.
NAME:
cast - Command line tool for the Chromecast
USAGE:
cast [global options] command [command options] [arguments...]
VERSION:
0.0.5

Thanks for highlighting this. It's been a while since a release binary was built - I'll get it building again and release the latest master.

Try: https://github.com/barnybug/go-cast/releases/tag/0.1.0

Yep that fixed it.  Thanks for you help.


Landing page: Add feedback button

https://phabricator.endlessm.com/T34665

Updated to the final URL and squashed fixups, this is ready for final review and merging.


[Bug]: CodeActAgent awaits user input after each step

Is there an existing issue for the same bug?

[X] I have checked the troubleshooting document at https://opendevin.github.io/OpenDevin/modules/usage/troubleshooting
[X] I have checked the existing issues.

Describe the bug
Currently, with the most recent master version, the agent seems to await user input after each step. Here is an example screenshot:

Current Version
commit 3d53d363b4416b05046dd390ca37d5745defce5a (origin/main, origin/HEAD, neubig/main, main)
Author: Boxuan Li<EMAIL_ADDRESS>Date:   Tue May 14 00:50:29 2024 -0700

Installation and Configuration
`make build; make run`

Model and Agent

Model: GPT-4
Agent: CodeAct

Reproduction Steps
No response
Logs, Errors, Screenshots, and Additional Context
In the logs, after every step I see:
08:30:01 - opendevin:INFO: agent_controller.py:189 - Setting agent(CodeActAgent) state from AgentState.RUNNING to AgentState.AWAITING_USER_INPUT

For instance, here is the first action:
08:56:13 - opendevin:INFO: codeact_agent.py:279 - Cost: 0.02 USD | Accumulated Cost: 0.02 USD
08:56:13 - ACTION
MessageAction(content="Let's start by cloning the repository and creating a new branch. I'll proceed with these steps first.", wait_for_response=True, action='message')
08:56:13 - opendevin:INFO: agent_controller.py:182 - Setting agent(CodeActAgent) state from AgentState.RUNNING to AgentState.AWAITING_USER_INPUT

add "Don't give me intermediate steps"  to the task

I'm not sure whether we can make CodeActAgent do like this : https://github.com/OpenDevin/OpenDevin/pull/1777
Let llm to choose whether continue executing or ask for user input ? @neubig

@assertion Based on my understanding of the solution in https://github.com/OpenDevin/OpenDevin/pull/1777, you are forcing the model to predict the argument of True vs. False and parse it to decide whether to ask for user input?
I think that eventually comes down to whether the LLM can choose between acting and asking for input. For GPT-4, this is probably not a problem - even if you don't force the model to generate arguments, they can still perform tasks autonomously vs. asking humans.
If you instruct a weaker model to do so, I think they will struggle with this nevertheless -- even if you force them to generate these "ask user input = True vs. False", plus we have to deal with parsing for this, which is kinda undesirable :(
I think we can probably address this by introducing the full-autonomous mode: https://github.com/OpenDevin/OpenDevin/issues/1798

@neubig Which version of GPT-4 are you running?

the agent seems to await user input after each step.

Nope, only when there is no coding action.


video not playing

Version
ExoPlayer 2.19.1
More version details
releaseVersion = '2.19.1'
releaseVersionCode = 2_019_001
Devices that reproduce the issue

vivo v23e 5G Android OS 11

Samsung M33 5G Android OS 13

One Plus 8T Android OS 13

Samsung Tab 3 Android OS 11

Devices that do not reproduce the issue
No response
Reproducible in the demo app?
Yes
Reproduction steps
Try playing the stream in the demo application.
Expected result
Media should play proerly
Actual result
Frames skipping
Media
https://customer-592a255a0tcjq2h1.cloudflarestream.com/c4001bd24ade448f8fb0b2265e651d7f/manifest/video.m3u8
Bug Report

[X] You will email the zip file produced by adb bugreport to<EMAIL_ADDRESS>after filing this issue.

@rohitjoins this one

@rohitjoins

I have created on both repo https://github.com/google/ExoPlayer/issues/11333

It's looks perfect.
https://github.com/androidx/media/assets/3471963/facb5dfb-b387-4194-b71c-2f38512deedb

10 seconds of playback won't show you issue switch to time lapse of 9.34 and play for 5 more mins you will see video frames skipping also if you have access to<EMAIL_ADDRESS>you can find my email where I have shared complete log report
Audio sink error
com.google.android.exoplayer2.audio.AudioSink$UnexpectedDiscontinuityException: Unexpected audio track timestamp discontinuity: expected<PHONE_NUMBER>400, got<PHONE_NUMBER>734
at com.google.android.exoplayer2.audio.DefaultAudioSink.handleBuffer(DefaultAudioSink.java:999)
at com.google.android.exoplayer2.audio.MediaCodecAudioRenderer.processOutputBuffer(MediaCodecAudioRenderer.java:709)
at com.google.android.exoplayer2.mediacodec.MediaCodecRenderer.drainOutputBuffer(MediaCodecRenderer.java:1889)
at com.google.android.exoplayer2.mediacodec.MediaCodecRenderer.render(MediaCodecRenderer.java:782)
at com.google.android.exoplayer2.ExoPlayerImplInternal.doSomeWork(ExoPlayerImplInternal.java:1007)
at com.google.android.exoplayer2.ExoPlayerImplInternal.handleMessage(ExoPlayerImplInternal.java:502)
at android.os.Handler.dispatchMessage(Handler.java:102)
at android.os.Looper.loopOnce(Looper.java:226)
at android.os.Looper.loop(Looper.java:313)
at android.os.HandlerThread.run(HandlerThread.java:67)

I think the exo team will says it's your video problem.

I think the exo team will says it's your video problem. Because i have experience.

But it's working fine on VLC Player so I don't think media is having issue.

@rohitjoins have you checked this yet?

Hi @maven08,
I haven't checked the media yet.

But it's working fine on VLC Player so I don't think media is having issue.

This is not necessarily true, we try and adhere to the spec when playing media. Other media players may have chosen other ways to fail or play media when it doesn't adhere to the spec which may not apply to ExoPlayer.


Add "are you sure" step to delete an alert

I pushed the "Delete alert" button by accident and my alert was gone immediately. I think a double check by "Are you sure?" step should be introduced. Removing an alert is not something a user does on daily basis, so he will not be concerned to be asked twice about a deletion all the time he/she wants to delete one.
@niconoe: do you agree?

Yeah, absolutely!

@damianooldoni: this has been implemented and should be visible on the dev website, can you have a look?

Thanks @niconoe. Yes, it works. Maybe adding a full stop (.) at the end of the sentence This can't be undone?

Good catch. This is now fixed!


extfrm using power threshold

Implement extfrm for twf and jnt extraction using power threshold parameter defined in speaker-dependent conf file

Coverage decreased (-0.2%) to 88.394% when pulling a148486644ea505256069fecf1f20b3f429cdabc on extfrm_power into 1216c5f9211f12de59b06c43db1be9d09b738f8d on master.


OSASINFRA-3546: docs/openstack: fix typo

What this PR does / why we need it:
Fix a typo that broke rendering.

/approve

/cc MaysaMacedo

/lgtm


CanonRFLensKit: Trio of Canon RF Zoom Lenses for Astrophotography

| Testing a Trio of Canon RF Zoom Lenses for Astrophotography  – The Amazing Sky |

#2377 closes #2522

#2377 closes #2522


Discord mentions do not work on webhooks

Describe the bug
A clear and concise description of what the bug is.
Following the merge of the Discord Mention PR, i discovered that the bridges i was operating that use webhooks on discord were not handling mentions
To Reproduce
Steps to reproduce the behavior:
Set up a standard discord with webhook) to anywhere bridge ad try to mention yourself
(i test with irc)
Expected behavior
A clear and concise description of what you expected to happen.
the webhooks sends proper mentions to the channel
Additional context
when running matterbridge in debug i (believe) i did not see any messages about trying out different mentions when parsing messages passing through, so i believe it might not even pass through the same code

Yes, webhooks take another path.
Should be fixed now.


Application Hangs when accessing the Large Bearer Token Value

I have checked the following:

[X] I use the newest version of bruno.
[X] I've searched existing issues and found nothing related to my issue.

Describe the bug
The application becomes slow when accessing any request which has long bearer tokens configures
.bru file to reproduce the bug
have a 1571 or longer JWT bearer token
Screenshots/Live demo link
This video does not have the mouse action in it, but it is considerably slow in comparision to other tools
https://github.com/user-attachments/assets/e0524494-ee49-4e4c-bbc1-286e4f9725ae

Hi @theinhumaneme, thanks for reporting, we're working on this!

Related #2842. We're tracking the issue here.

@theinhumaneme We are prioritising this issue and are looking into this week.
I deleted the video in your description as I felt it had some sensitive tokens.
We are tracking the issue here - #2842 Hence closing this as a duplicate.

Thank you team, could you please update this comment thread when the issue is fixed? Until then I'll be using postman although, I love Bruno's git integration :)

Hi @theinhumaneme, this issue was resolved and released yesterday. Please check it out!

thank you @sreelakshmi-bruno


Code Generation using Annotation Processors (JSR 269)

I would like to propose a feature:
It would be extremely nice if TypeScript compiler allowed code generation and annotation processing similar to JSR 269.
Code in TS relies on runtime reflection for everything. If some of the analysis done at compile time and properly statically linked code generated, it would have huge performance increase during run time.
For example of successful implementation of this approach look and Google's Dagger 2.

With issue https://github.com/Microsoft/TypeScript/issues/5595 in place, a lot of these scenarios should be enabled.


bobs and angels playthrough

hello i am playing bobs and angels seablock
and with my power i have an issue
it seems when certain machines hit max level they change to a whole different machine
for example an liquefier gets changed to an ore crusher

I am also having the exact same issue with the Liquifier.
Took me a while to work out why all of them were replaced with Ore Crushers...
The Ore Crushers level up completely fine however, just seems like that machine so far for me

Yeah it seems like that but with the latest update of 0.6.8 of this mod it says in the changelog that the machines dont auto promote to a mk2 machine but it still does that

Yep,
Auto-levels up for me too.
I missed that part of the changelog, there doesn't seem to be anything within game settings for enabling/disabling that either

["liquifier"] = {
name = "liquifier",
level_name = "liquifier-level-",
max_level = 25,
next_machine = "ore-crusher"
},
["liquifier-2"] = {
name = "liquifier-2",
level_name = "liquifier-2-level-",
max_level = 25,
next_machine = "ore-crusher-2"
},
["liquifier-3"] = {
name = "liquifier-3",
level_name = "liquifier-3-level-",
max_level = 50,
next_machine = "ore-crusher-3"
},
["liquifier-4"] = {
name = "liquifier-4",
level_name = "liquifier-4-level-",
max_level = 100
},
found the issue it's in the control.loua
["liquifier"] = {
name = "liquifier",
level_name = "liquifier-level-",
max_level = 25,
next_machine = "liquifier-2"
},
["liquifier-2"] = {
name = "liquifier-2",
level_name = "liquifier-2-level-",
max_level = 25,
next_machine = "liquifier-3"
},
["liquifier-3"] = {
name = "liquifier-3",
level_name = "liquifier-3-level-",
max_level = 50,
next_machine = "liquifier-4"
},
["liquifier-4"] = {
name = "liquifier-4",
level_name = "liquifier-4-level-",
max_level = 100
},
This should be the fix

Thank you very much, released in 0.6.9


rmw: 7.4.2-1 in 'rolling/distribution.yaml' [bloom]

Increasing version of package(s) in repository rmw to 7.4.2-1:

upstream repository: https://github.com/ros2/rmw.git
release repository: https://github.com/ros2-gbp/rmw-release.git
distro file: rolling/distribution.yaml
bloom version: 0.12.0
previous version for package: 7.4.1-1

rmw
* removed deprecated rmw_node_assert_liveliness() (#373 <https://github.com/ros2/rmw/issues/373>)
* Contributors: Alejandro Hernández Cordero

rmw_implementation_cmake

No changes

Holding for an upcoming Rolling sync.


swapped real and imaginary parts of extended complex xdotu and xdotc results

Hi,
Please, could you check if the extended complex dot-product functions xdotu and xdotc produce their results with the real and imaginary parts swapped?
Here is a test code (Fortran 90, gfortran 5.2 on MacOS 10.11.3), to be linked with OpenBLAS:
program qxtest

  implicit none

  integer, parameter :: n = 2, incx = 1, incy = 1

  complex(8) :: zx(n), zy(n), zc, zu
  complex(10) :: xx(n), xy(n), xc, xu

  complex(8), external :: zdotc, zdotu
  complex(10), external :: xdotc, xdotu

  zx(1) = ( 1e0_8, -1e0_8)
  zx(2) = (-3e0_8,  3e0_8)
  zy(1) = ( 2e0_8, -2e0_8)
  zy(2) = ( 4e0_8, -4e0_8)

  xx(1) = ( 1e0_10, -1e0_10)
  xx(2) = (-3e0_10,  3e0_10)
  xy(1) = ( 2e0_10, -2e0_10)
  xy(2) = ( 4e0_10, -4e0_10)

  zc = zdotc(n, zx, incx, zy, incy)
  print *, 'dotc(KIND= 8) =', zc
  xc = xdotc(n, xx, incx, xy, incy)
  print *, 'dotc(KIND=10) =', xc

  zu = zdotu(n, zx, incx, zy, incy)
  print *, 'dotu(KIND= 8) =', zu
  xu = xdotu(n, xx, incx, xy, incy)
  print *, 'dotu(KIND=10) =', xu

end program qxtest

And the output is:
 dotc(KIND= 8) = ( -20.000000000000000     ,  0.0000000000000000     )
 dotc(KIND=10) = (  0.00000000000000000000      , -20.0000000000000000000      )
 dotu(KIND= 8) = (  0.0000000000000000     ,  20.000000000000000     )
 dotu(KIND=10) = (  20.0000000000000000000      ,  0.00000000000000000000      )

So, double precision functions return the correct results, but the extended precision ones do not.

how did you build openblas to have x* functions exported? normally they are not.

Are you sure it is the correct kind, if I remember correctly gcc provides 2
kinds of same bit length.
I am not at my box but run through kinds to check which it actually is.
--
Kind regards Nick Papior
On 31 Jan 2016 02:51, "brada4"<EMAIL_ADDRESS>wrote:

how did you build openblas to have x* functions exported? normally they
are not.
—
Reply to this email directly or view it on GitHub
https://github.com/xianyi/OpenBLAS/issues/768#issuecomment-177356953.

@brada4:
I build from the latest GitHub sources.
Here's my Makefile.rule:
#
#  Beginning of user configuration
#

# This library's version
VERSION = 0.2.16.dev

# If you set the suffix, the library name will be libopenblas_$(LIBNAMESUFFIX).a
# and libopenblas_$(LIBNAMESUFFIX).so. Meanwhile, the soname in shared library
# is libopenblas_$(LIBNAMESUFFIX).so.0.
# LIBNAMESUFFIX = omp

# You can specify the target architecture, otherwise it's
# automatically detected.
TARGET = HASWELL

# If you want to support multiple architecture in one binary
# DYNAMIC_ARCH = 1

# C compiler including binary type(32bit / 64bit). Default is gcc.
# Don't use Intel Compiler or PGI, it won't generate right codes as I expect.
CC = clang -ffp-contract=on -integrated-as

# Fortran compiler. Default is g77.
FC = gfortran -Wa,-q -Wl,-no_compact_unwind

# Even you can specify cross compiler. Meanwhile, please set HOSTCC.

# cross compiler for Windows
# CC = x86_64-w64-mingw32-gcc
# FC = x86_64-w64-mingw32-gfortran

# cross compiler for 32bit ARM
# CC = arm-linux-gnueabihf-gcc
# FC = arm-linux-gnueabihf-gfortran

# cross compiler for 64bit ARM
# CC = aarch64-linux-gnu-gcc
# FC = aarch64-linux-gnu-gfortran

# If you use the cross compiler, please set this host compiler.
# HOSTCC = gcc

# If you need 32bit binary, define BINARY=32, otherwise define BINARY=64
BINARY=64

# About threaded BLAS. It will be automatically detected if you don't
# specify it.
# For force setting for single threaded, specify USE_THREAD = 0
# For force setting for multi  threaded, specify USE_THREAD = 1
USE_THREAD = 1

# If you're going to use this library with OpenMP, please comment it in.
# USE_OPENMP = 1

# You can define maximum number of threads. Basically it should be
# less than actual number of cores. If you don't specify one, it's
# automatically detected by the the script.
# NUM_THREADS = 4

# if you don't need to install the static library, please comment it in.
# NO_STATIC = 1

# if you don't need generate the shared library, please comment it in.
# NO_SHARED = 1

# If you don't need CBLAS interface, please comment it in.
NO_CBLAS = 1

# If you only want CBLAS interface without installing Fortran compiler,
# please comment it in.
# ONLY_CBLAS = 1

# If you don't need LAPACK, please comment it in.
# If you set NO_LAPACK=1, the library automatically sets NO_LAPACKE=1.
NO_LAPACK = 1

# If you don't need LAPACKE (C Interface to LAPACK), please comment it in.
# NO_LAPACKE = 1

# If you want to use legacy threaded Level 3 implementation.
# USE_SIMPLE_THREADED_LEVEL3 = 1

# If you want to drive whole 64bit region by BLAS. Not all Fortran
# compiler supports this. It's safe to keep comment it out if you
# are not sure(equivalent to "-i8" option).
# INTERFACE64 = 1

# Unfortunately most of kernel won't give us high quality buffer.
# BLAS tries to find the best region before entering main function,
# but it will consume time. If you don't like it, you can disable one.
NO_WARMUP = 1

# If you want to disable CPU/Memory affinity on Linux.
NO_AFFINITY = 1

# if you are compiling for Linux and you have more than 16 numa nodes or more than 256 cpus
# BIGNUMA = 1

# Don't use AVX kernel on Sandy Bridge. It is compatible with old compilers
# and OS. However, the performance is low.
# NO_AVX = 1

# Don't use Haswell optimizations if binutils is too old (e.g. RHEL6)
# NO_AVX2 = 1

# Don't use parallel make.
# NO_PARALLEL_MAKE = 1

# Force number of make jobs. The default is the number of logical CPU of the host.
# This is particularly useful when using distcc
# MAKE_NB_JOBS = 8

# If you would like to know minute performance report of GotoBLAS.
# FUNCTION_PROFILE = 1

# Support for IEEE quad precision(it's *real* REAL*16)( under testing)
# QUAD_PRECISION = 1

# Theads are still working for a while after finishing BLAS operation
# to reduce thread activate/deactivate overhead. You can determine
# time out to improve performance. This number should be from 4 to 30
# which corresponds to (1 << n) cycles. For example, if you set to 26,
# thread will be running for (1 << 26) cycles(about 25ms on 3.0GHz
# system). Also you can control this mumber by THREAD_TIMEOUT
# CCOMMON_OPT	+= -DTHREAD_TIMEOUT=26

# Using special device driver for mapping physically contigous memory
# to the user space. If bigphysarea is enabled, it will use it.
# DEVICEDRIVER_ALLOCATION = 1

# If you need to synchronize FP CSR between threads (for x86/x86_64 only).
CONSISTENT_FPCSR = 1

# If any gemm arguement m, n or k is less or equal this threshold, gemm will be execute
# with single thread. You can use this flag to avoid the overhead of multi-threading
# in small matrix sizes. The default value is 4.
# GEMM_MULTITHREAD_THRESHOLD = 4

# If you need santy check by comparing reference BLAS. It'll be very
# slow (Not implemented yet).
# SANITY_CHECK = 1

# Run testcases in utest/ . When you enable UTEST_CHECK, it would enable
# SANITY_CHECK to compare the result with reference BLAS.
# UTEST_CHECK = 1

# The installation directory.
PREFIX = /usr/local

# Common Optimization Flag;
# The default -O2 is enough.
COMMON_OPT = -O3 -fexceptions -march=native -pipe

# gfortran option for LAPACK
# enable this flag only on 64bit Linux and if you need a thread safe lapack library
FCOMMON_OPT = -frecursive

# Profiling flags
COMMON_PROF = -pg

# Build Debug version
# DEBUG = 1

# Set maximum stack allocation.
# The default value is 2048. 0 disable stack allocation a may reduce GER and GEMV
# performance. For details, https://github.com/xianyi/OpenBLAS/pull/482
#
# MAX_STACK_ALLOC = 0

# Add a prefix or suffix to all exported symbol names in the shared library.
# Avoid conflicts with other BLAS libraries, especially when using
# 64 bit integer interfaces in OpenBLAS.
# For details, https://github.com/xianyi/OpenBLAS/pull/459
#
# The same prefix and suffix are also added to the library name,
# i.e. you get lib$(SYMBOLPREFIX)openblas$(SYMBOLSUFFIX) rather than libopenblas
#
# SYMBOLPREFIX=
# SYMBOLSUFFIX=

#
#  End of user configuration
#

@zerothi:
I'm reasonably sure it is the correct kind, please see:
program zxkind

  implicit none

  complex(10) :: a
  real(10) :: b(2)
  equivalence (a,b(1))

  a = (1e0_10, 2e0_10)

  print *, 'sizeof(real(10))    =', sizeof(b(1))
  print *, 'sizeof(complex(10)) =', sizeof(a)
  print *, 'first component     =', b(1)
  print *, 'second component    =', b(2)

end program zxkind

The output (gfortran 5.2, MacOS):
 sizeof(real(10))    =                   16
 sizeof(complex(10)) =                   32
 first component     =   1.00000000000000000000
 second component    =   2.00000000000000000000

Sizeof real extended is 16 bytes (10 bytes + padding), sizeof complex extended is twice that, and the first 16 bytes of a complex variable represents its real part, the last 16 bytes its imaginary part.

x* functions are not present in current version. They were in gotoblas, but probably never worked at all, nor there is any standard behaviour defined for them...
can you quickly check with
nm -B libopenblas.dylib (should have no xdotu)
ldd your_testcase (does it really link to the right openblas, or you have some old version around)

From the NO_LAPACK in your Makefile.rule I suspect you are linking to a netlib(-derived) LAPACK that may have been built with USE_XBLAS, pulling in netlib xblas (libxblas) which is a separate project.

@brada4:
nm -m libopenblas_haswell-r0.2.16.dev.dylib | grep xdot
0000000000025010 (__TEXT,__text) external _xdotc_
0000000000bf58e0 (__TEXT,__text) non-external (was a private external) _xdotc_k
0000000000025060 (__TEXT,__text) external _xdotu_
0000000000bf5a60 (__TEXT,__text) non-external (was a private external) _xdotu_k

The initial example (qxtest) is built as:
gfortran -O3 -fexceptions -frecursive -march=native -pipe -Wa,-q qxtest.f90 -L/usr/local/lib -lopenblas

(Using the same compiler flags as for the library itself.)
And yes, I link to this version (symlinked as libopenblas.dylib):
$ ls -la /usr/local/lib/libopenblas.dylib
lrwxr-xr-x  1 root  admin  38 30 Jan 23:04 /usr/local/lib/libopenblas.dylib -> libopenblas_haswellp-r0.2.16.dev.dylib
$ otool -L ./a.out
./a.out:
	/usr/local/lib/libopenblas_haswellp-r0.2.16.dev.dylib (compatibility version 0.0.0, current version 0.0.0)
	/usr/local/gfortran/lib/libgfortran.3.dylib (compatibility version 4.0.0, current version 4.0.0)
	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1226.10.1)
	/usr/local/gfortran/lib/libgcc_s.1.dylib (compatibility version 1.0.0, current version 1.0.0)
	/usr/local/gfortran/lib/libquadmath.0.dylib (compatibility version 1.0.0, current version 1.0.0)

(As you might have noticed, this is MacOS, not Linux.)
@martin-frbg:
In this example, I don't link to any LAPACK implementation whatsoever, and certainly not to Xblas, which I have never used.
And I deliberately don't want to build LAPACK, CBLAS interface, etc. included in OpenBLAS.
I have an unmodified, up-to-date repository of OpenBLAS, of which I make a copy, replace Makefile.rule with the one included above, and say "make" and "sudo make install".
What I get is a library that most certainly contains the x* routines.
I don't link to GotoBLAS, Xblas, or anything of the sort, as you may see above.

Got it now - these functions are only built if EXPRECISION is defined, or conversely if NO_EXPRECISION is not set in Makefile.system. For all OSNAMEs except Darwin, AIX and the BSDs, NO_EXPRECISION is explicitly set in Makefile.system, so presumably only the latter are assumed to have proper system library support for the extended complex functions. (Given that there is an open ticket for updating the build system to support AIX, it seems doubtful that it has been tested on that platform at least in recent years. Perhaps the commit history of Makefile.system will reveal if the entries were actually adapted or just copied over from existing ones when new operating systems were added.)

Haswell Linux build (after removing the NO_EXPRECISION) gives exact same result for the testcase as above. Most of the settings in Makefile.system appear to be unchanged from GotoBLAS2, but the NO_EXPRECISION entry for Linux was added by @wernsaar in May 2014 (a35a1a9) with a commit message reading just "changed makefiles for lapack development".

Both xdotc and xdotu are compiled from kernel/x86_64/zdot.S (with -DCONJ for xdotc, -UCONJ for xdotu, -DCOMPLEX -DXDOUBLE for both), this file is unchanged from GotoBLAS2. Perhaps the single "#ifndef CONJ" before the regular return from the function simply needs to be inverted - no other function appears to be built from this assembler file ?

Is it float(10) or float(12) or float(16) (looking around #ifdefs)
https://www.freebsd.org/cgi/man.cgi?query=ieee

@martin-frbg:
Thanks for looking into that!
I'm not versed in x87 assembler, either.
If there's no one still maintaining that code, maybe I'd dare to try fixing it, but would otherwise leave that to someone more knowledgable.
@brada4:
I'd be very cautious in trying to follow that FreeBSD's manpage.
From what I've tried so far, on 64-bit Linux, Darwin, and Windows (with Intel's C compiler), 80-bit extended precision type is supported, in C as long double, in GNU Fortran as real(10), and stored as 16 bytes, of which the upper 6 bytes are unused.
Therefore, what that manpage says about storing extended floats into 12 bytes is puzzling me.
I don't have any FreeBSD installation around, but do have an OpenBSD 5.8 64-bit, and it also uses 16 bytes per a long double.

I would say that this xdot kernel is still being compiled by accident, just disable it. There is no chance it will ever work on all systems it kind of compiles for.
PS Darwin math library is from FreeBSD...

Just swap the result at the end of kernel.

@xianyi:
Thank you!
I see the correct results now, and will close the issue.


enemy POIs get snapped to world.y = 0

Exposed by trying vertical jumping levels, when enemies drop a new POI, the POI get snapped to Y=0.0.
https://github.com/ioxu/key/blob/b0e1602a8ca28ffff386a5155ded98fa205f1027/data/enemies/Enemy.gd#L497

		#poi.transform.origin = position * Vector3(1.0, 0.0, 1.0)
		poi.transform.origin = position

Good enough for now.


if transaction status non-success, bypass intercept plugin

Don't allow the generator or statichit plugin to intercept the transaction
if a prior plugin has changed the transaction status to something non-success.

Cherry-picked to v9.1.x branch.


Check that files are not half downloaded

If the offlinedatasci process is stopped halfway through downloading a file, running it a second time doesn't cause the rest of the file to be downloaded. There needs to be a check of each file's downloaded size and the size of source file to see if it the download should be redone (or better still continued).

wget calls are using -c so continuation should work, will check

@abhidg - yes, that's true for lessons, but not for other things. @garezana & I have discussed plans for not writing incomplete downloads and retrying (see also #46) that should work for everything except miniCRAN.


[Demo] Aliases are bleeding out of the demo section

This is a followup to:

#3714

In particular because some of the configured aliases are bleeding out of the demo section, which we don't want. For example:

https://opentelemetry.io/docs/feature_flags redirects into the Demo section at https://opentelemetry.io/docs/demo/feature-flags/. IMHO, we don't want this.

The redirect mentioned above is due to https://github.com/open-telemetry/opentelemetry.io/pull/3714#discussion_r1457320442.
cc @flands @cartermp

Oh, it's not as bad as I thought for the bleeding: only one page is bleeding out of the demo section. But the other aliases that were added are still wrong. I'll fix those too.


find all repo packages

move the logic to find the repo's project packages into one location (fix https://github.com/flutter/flutter/issues/3359)
use it from update-packages and analyze
fix some issues in dev/benchmarks/complex_layout and examples/hello_android/app/src/flutter

fix https://github.com/flutter/flutter/issues/3157

This is great.

Happy to take another look if you decide to change this substantially based on my last comment above, but either way is fine.

I updated this to use a marker file (.dartignore). It doesn't have any content - unlike .gitignore files. If this works out I'll roll the same logic into the atom plugin.

LGTM!


WIP: Procedure for creating disk images of NLF data

TO DO:

tests
make DAG file more readable by moving tasks and task groups to a separate file?

I would say this is now ready for an "intermediate review". Tests and some final refactoring are definitely still missing, but the actual functionality is there.
I tested the initial download / update download DAG runs by simulating a situation where I first downloaded data in col-361 from before 2020-06-06, and then set the "previous DAG run" into that date and downloaded whatever data was new after that as an update download. It worked with no issues. @aajarven

Suggested changes are implemented, ready for final blessings.

I did my best to improve readability of our task groups by defining the tasks as new custom operators. One of them is still quite heavy, but at least it is not in the same file as the task group definition with million levels of indentation.
I also used partial and extend when creating batch download tasks, since this avoids looping. I have a hunch that using this task mapping technique may make the DAG slightly less heavy and resource-syöppö.


darg and drop fail when it is the first line

Closing this as it looks like a duplicate of #145, but feel free to reopen if not! :)


Address #105 by implementing eraseToAnyWorkflow

Linked Issue: Closes #105

Checklist:

[ ] Did you sign the Contributor License Agreement?
[ ] Is the linter reporting 0 errors?
[ ] Did you comply with our styleguide?
[ ] Is there adequate test coverage for your new code?
[ ] Does the CI pipeline pass?
[ ] Did you update the documentation?
[ ] Did you change the public API?
[ ] Have you done everything you can to make sure that errors that can occur are compile-time errors, and if they have to be runtime do you have adequate tests and documentation around those runtime errors? For more details.

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.

Codecov Report

Merging #168 (9ca8555) into main (3097cea) will increase coverage by 0.00%.
The diff coverage is 100.00%.

:exclamation: Current head 9ca8555 differs from pull request most recent head abdc59f. Consider uploading reports for the commit abdc59f to get more accurate results

@@           Coverage Diff           @@
##             main     #168   +/-   ##
=======================================
  Coverage   91.27%   91.28%
=======================================
  Files          92       92
  Lines        2373     2375    +2
=======================================
+ Hits         2166     2168    +2
  Misses        207      207

Impacted Files
Coverage Δ

Sources/SwiftCurrent/Models/Workflow.swift
100.00% <100.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update bc28d62...abdc59f. Read the comment docs.