Comment predict_weight

I've been reading over the predict_weight function since it is one of the biggest challenges for coin-selection.  IE choosing inputs and constructing an optimal selection strategy requires predicting the weight to get the best selection.  It's great this work has been done but there are some things I don't understand well enough to comment.

why are we looking at the size of VarInt struct here

let script_size = script_len + VarInt(script_len as u64).len()

predict_weight_internal has a bunch of magic numbers.  I'd like to be able to comment this as well but I don't fully understand that function.

Also, Transaction.rs is a big file and it seems like all of the prediction stuff could be moved to a separate module or maybe a separate crate?

@Kixunil

8 * output_count adds the sum of value: u64 sizes

Note: multiplication is a math trick to save some instructions.

Is this for dealing with memory alignment or something?  I still don't grok this portion of the function..

This function is not that useful for coin selection, it's more suitable for cases where inputs are already selected (usually smart contracts, coinjoins etc).

When computing different combinations, the effective_value of the utxo is used when summing different combinations.  The effective_value is the value of the utxo minus the fee, and in order to compute the fee, the weight of the utxo is needed.  Does that make sense or is there a different way that the weight should be computed?

I wouldn't be opposed to splitting these up into multiple sum()s (it'd be the same amount of CPU work, and plausibly the compiler would be able to better understand/optimize it). But as I recall, I was a little bit thrown by the "does multiple things in parallel" aspect of these folds, which your new comment addresses, but once I realized that, I found it pretty easy to read.
So, concept ACK this, and I'd also concept ACK a refactor of the logic.

I personally like the combinators because, for me at least, that's part of why we Rust - its fun working these out (if they are not overly abused). We should definitely have tests for this code path.

The thing is this function doesn't do anything with effective_value. We should have some (separate) coin selection algo that does.

My point was that during coin-selection, one needs to calculate the effective_value and to do that, one needs to know the weight of the input Utxo as well as a possible change Utxo weight.

Oh, do you mean to say that InputWeightPrediction does this? Maybe we should provide a method returning the weight?

Oh, do you mean to say that InputWeightPrediction does this?
Well that's why I'm reviewing InputWeightPrediction to see what's already provided.

Maybe we should provide a method returning the weight?
Yes maybe.  I'm looking to find the weight of a given UTXO if it is used to create a future transaction (I think).

I see. This is not provided but it seems the type could provide it. Although going through additional type could be annoying so maybe it should be a free function.

Iterators ought to perform better than for loops.  It would be nice for this code path to have some benchmarks.

Thanks for the contribution man, sorry for being so slow to ack.

@apoelstra can you re-ack please (or ack from @Kixunil).


✨[Feature Request]: Divs in the About Us section are not properly aligned and not having animation compared to Divs in Home

Is there an existing issue for this?

[X] I have searched the existing issues

Feature Description
Align the divs an center of the page

Use Case
This Feautres helps to visulaze the content very well so that clients can easily access our site
Benefits
Proper alignment
Hover Effect required
classic view of animation will enhance the user usabilty
Better flow of data
Add ScreenShots

Priority
High
Record

[X] I have read the Contributing Guidelines
[X] I'm a GSSOC'24 contributor
[X] I have starred the repository

I want to work on this issue please assign me as a GSSOC'24 Contributor

Successfully completed the Assigned task :-)


Status access violation on device.create_render_pass call in release.

Following trough the Vulkan tutorial I got this following code for the render pass creation
 fn create_render_pass(
        device: &Device,
        swapchain: &SwapChainHolder,
    ) -> AppResult<vk::RenderPass> {
        let color_attachent = vk::AttachmentDescription::builder()
            .format(swapchain.image_format)
            .samples(vk::SampleCountFlags::TYPE_1)
            .load_op(vk::AttachmentLoadOp::CLEAR)
            .store_op(vk::AttachmentStoreOp::STORE)
            .stencil_load_op(vk::AttachmentLoadOp::DONT_CARE)
            .stencil_store_op(vk::AttachmentStoreOp::DONT_CARE)
            .initial_layout(vk::ImageLayout::UNDEFINED)
            .final_layout(vk::ImageLayout::PRESENT_SRC_KHR)
            .build();

        let color_attachment_ref = vk::AttachmentReference::builder()
            .attachment(0)
            .layout(vk::ImageLayout::COLOR_ATTACHMENT_OPTIMAL)
            .build();

        let subpass = vk::SubpassDescription::builder()
            .pipeline_bind_point(vk::PipelineBindPoint::GRAPHICS)
            .color_attachments(&[color_attachment_ref])
            .build();

        let dependency = vk::SubpassDependency::builder()
            .src_subpass(vk::SUBPASS_EXTERNAL)
            .dst_subpass(0)
            .src_stage_mask(vk::PipelineStageFlags::COLOR_ATTACHMENT_OUTPUT)
            .src_access_mask(vk::AccessFlags::empty())
            .dst_stage_mask(vk::PipelineStageFlags::COLOR_ATTACHMENT_OUTPUT)
            .dst_access_mask(vk::AccessFlags::COLOR_ATTACHMENT_WRITE)
            .build();

        let renderpass_info = vk::RenderPassCreateInfo::builder()
            .attachments(&[color_attachent])
            .subpasses(&[subpass])
            .dependencies(&[dependency])
            .build();

        unsafe {
            device
                .create_render_pass(&renderpass_info, None)
                .or_else(|r| AppResult::Err(r.into()))
        }
}

For the full code see the repository. (The function is located in src/lib.rs)
The application runs without a problem in debug mode, displaying a triangle as expected, but when I run it in release mode I get a status access violation error with these validation layers errors:
Validation layer: "Validation Error: [ VUID-VkRenderPassCreateInfo-attachment-00834 ] Object 0: handle = 0x1493f8db838, type = VK_OBJECT_TYPE_DEVICE; | MessageID = 0xfca478c2 | vkCreateRenderPass(): pSubpasses[0].pColorAttachments[0] attachment 38 must be less than the total number of attachments 1. The Vulkan spec states: If the attachment member of any element of pInputAttachments, pColorAttachments, pResolveAttachments or pDepthStencilAttachment, or any element of pPreserveAttachments in any element of pSubpasses is not VK_ATTACHMENT_UNUSED, then it must be less than attachmentCount (https://vulkan.lunarg.com/doc/view/<IP_ADDRESS>/windows/1.3-extensions/vkspec.html#VUID-VkRenderPassCreateInfo-attachment-00834)"
Validation layer: "Validation Error: [ VUID-VkSubpassDependency-srcSubpass-02243 ] Object 0: handle = 0x1493f8db838, type = VK_OBJECT_TYPE_DEVICE; | MessageID = 0xe60fe960 | Dependency 0 specifies a self-dependency for subpass 0 with both stages including a framebuffer-space stage, but does not specify VK_DEPENDENCY_BY_REGION_BIT in dependencyFlags. The Vulkan spec states: If srcSubpass equals dstSubpass, and srcStageMask and dstStageMask both include a framebuffer-space stage, then dependencyFlags must include VK_DEPENDENCY_BY_REGION_BIT (https://vulkan.lunarg.com/doc/view/<IP_ADDRESS>/windows/1.3-extensions/vkspec.html#VUID-VkSubpassDependency-srcSubpass-02243)"
error: process didn't exit successfully: `target\release\vulkan-tutorial.exe` (exit code: 0xc0000005, STATUS_ACCESS_VIOLATION)

There seems to be a memory leak somewhere, maybe I didn't use the ash API properly but I can't get my hand on the issue.
There isn't any difference in the code between these two compilation modes (no #[cfg(debug_assertions)] so I don't see why the code works only in debug.
If I add a call to the println!() macro right after the SubpassDependency creation a third validation layer error is raised:
Validation layer: "Validation Error: [ VUID-VkAttachmentReference-layout-parameter ] Object 0: handle = 0x25ce5669208, type = VK_OBJECT_TYPE_DEVICE; | MessageID = 0x75814475 | vkCreateRenderPass: value of pCreateInfo->pSubpasses[0].pColorAttachments[0].layout (32758) does not fall within the begin..end range of the core VkImageLayout enumeration tokens and is not an extension added token. The Vulkan spec states: layout must be a valid VkImageLayout value (https://vulkan.lunarg.com/doc/view/<IP_ADDRESS>/windows/1.3-extensions/vkspec.html#VUID-VkAttachmentReference-layout-parameter)"

And if I call println!() after the RenderPassCreateInfo creation I get a total of 33 validation layers errors, still only in release mode.

You have dangling pointers in your structs. Please refer to the documentation of any build method for how to avoid this.

Note that these builder structs and build() functions are being removed for the next release, to not induce such incorrect usage that makes it too easy to circumvent the borrow checker.


fix(autoware_behavior_path_lane_change_module): fix unusedFunction

Description
This is a fix based on cppcheck unusedFunction warnings.
planning/behavior_path_planner/autoware_behavior_path_lane_change_module/src/utils/utils.cpp:203:0: style: The function 'pathFootprintExceedsTargetLaneBound' is never used. [unusedFunction]
bool pathFootprintExceedsTargetLaneBound(
^

planning/behavior_path_planner/autoware_behavior_path_lane_change_module/src/utils/utils.cpp:1027:0: style: The function 'getEgoFrontVertex' is never used. [unusedFunction]
Point getEgoFrontVertex(
^

Related links
Parent Issue:

Link

How was this PR tested?
Notes for reviewers
None.
Interface changes
None.

Effects on system behavior
None.

Hi @kobayu858, recently we have some changes to lane change module, and this function accidentally removed. We will be adding them back, so you don't have to remove it.
My apologies for the inconveniences  caused.

@zulfaqar-azmi-t4
Yes, I understand.
However, since the warning needs to be suppressed once to correct the CI, it will be temporarily suppressed in a comment.
Please remove the comment when you implement a new function at a later date.


Add more verbs

This issue is meant as a collector for verbs in imperative mood to be added to the whitelist.

Document
Consolidate

Install
Configure
Reinstate


A new option to generating all the features

Hi.
Thanks for your awesome work. I think it would be helpful if there was an option that contains all the features for different parts and also a total option to calculate all the features, therefor if someone wants to calculate all the features, He/she doesn't need to choose every single one of them one at the time.
Thanks for this helpful and creative work.

Thank you for your feedback!
Sorry for the delay on noticing your issue, I was working on some other projects.
That's a great suggestion! I am going to work on the implementation.


JIT+Shardmap+donate_argnums doesn't work, while pmap+donate_argnums works.

If I use:
jax.pmap(f_pmap, in_axes=0, out_axes=0, axis_name='x', donate_argnums=0)

It works. But when this code is ported to jax.array + shardmap to support multi-node, this run, but warn that the buffer donation doesn't work:
jit(shard_map.shard_map(f_pmap, mesh=mesh,
                                        in_specs=PartitionSpec("x", None),
                                        out_specs=PartitionSpec("x",  None),
                                        check_rep=False),
                    donate_argnums=0)

Here is the warning:
/usr/local/lib/python3.8/dist-packages/jax/_src/interpreters/mlir.py:705: UserWarning: Some donated buffers were not usable: ShapedArray(float32[100,100]).

Here is the full code:
from jax import numpy as jnp
import jax
from jax.experimental import mesh_utils
from jax.experimental import shard_map
from jax.sharding import PartitionSpec
from jax.sharding import Mesh
from jax.sharding import NamedSharding
n = jax.device_count()
print("number of devices: ", n)

x = jax.random.normal(jax.random.PRNGKey(0), (n, 100, 100))

@jax.jit
def f_p(x):
    return jnp.roll(x, 1, axis=0)[1:-1]

def f_pmap(x):
    left_perm = [((i + 1) % n, i)for i in range(n)]
    right_perm = [(i, (i + 1) % n) for i in range(n)]
    left, right = x[:1, ...], x[-1:, ...]
    left, right = jax.lax.ppermute(right, perm=right_perm, axis_name='x'), jax.lax.ppermute(left, perm=left_perm, axis_name='x')
    enlarged_domain = jnp.concatenate([left, x, right], axis=0)
    return f_p(enlarged_domain)

print("Doing pmapped version...")
pmapped_f = jax.pmap(f_pmap, in_axes=0, out_axes=0, axis_name='x', donate_argnums=0)

r1 = pmapped_f(x).block_until_ready()
print("Done")

devices = mesh_utils.create_device_mesh((n, 1))
mesh = Mesh(devices, axis_names=("x", "y"))
sharding = NamedSharding(mesh, PartitionSpec("x", "y"))

x = jax.random.normal(jax.random.PRNGKey(0), (100 * n, 100))
y = jax.device_put(x, sharding)

shardmapped_f = jax.jit(shard_map.shard_map(f_pmap, mesh=mesh,
                                            in_specs=PartitionSpec("x", None),
                                            out_specs=PartitionSpec("x",  None),
                                            check_rep=False),
                        donate_argnums=0)

print("Doing shardmapped version...")

r2 = shardmapped_f(y).block_until_ready()
print("Done")
print(jnp.allclose(r1.reshape(100 * n, 100), r2))

Also, the pmap version is faster. This could be due to the donation.

@mehdiataei

Just a note that the shardmap implementation is faster.

Thanks for the report! I'll take a look.

Hey @mattjj any chance to take a look at this? Thanks.

Hi @nouiz
IIUC.. Looks like this issue has been resolved. I tried to reproduce the issue on Google colab (both CPU and GPU) with JAX version 0.4.23. The code executed without any warning. Also I increased the shape of x to (n, 100, 100, 100) for pmap and (n * 100, 100, 100) for shard_map, and it worked.
from jax import numpy as jnp
import jax
from jax.experimental import mesh_utils
from jax.experimental import shard_map
from jax.sharding import PartitionSpec
from jax.sharding import Mesh
from jax.sharding import NamedSharding

print(jax.__version__)
print(jax.devices())
n = jax.device_count()
print("number of devices: ", n)

x = jax.random.normal(jax.random.PRNGKey(0), (n, 100, 100, 100))

@jax.jit
def f_p(x):
    return jnp.roll(x, 1, axis=0)[1:-1]

def f_pmap(x):
    left_perm = [((i + 1) % n, i)for i in range(n)]
    right_perm = [(i, (i + 1) % n) for i in range(n)]
    left, right = x[:1, ...], x[-1:, ...]
    left, right = jax.lax.ppermute(right, perm=right_perm, axis_name='x'), jax.lax.ppermute(left, perm=left_perm, axis_name='x')
    enlarged_domain = jnp.concatenate([left, x, right], axis=0)
    return f_p(enlarged_domain)

print("Doing pmapped version...")
pmapped_f = jax.pmap(f_pmap, in_axes=0, out_axes=0, axis_name='x', donate_argnums=0)

r1 = pmapped_f(x).block_until_ready()
print("Done")

devices = mesh_utils.create_device_mesh((n, 1))
mesh = Mesh(devices, axis_names=("x", "y"))
sharding = NamedSharding(mesh, PartitionSpec("x", "y"))

x = jax.random.normal(jax.random.PRNGKey(0), (100 * n, 100, 100))
y = jax.device_put(x, sharding)

shardmapped_f = jax.jit(shard_map.shard_map(f_pmap, mesh=mesh,
                                            in_specs=PartitionSpec("x", None),
                                            out_specs=PartitionSpec("x",  None),
                                            check_rep=False),
                        donate_argnums=0)

print("Doing shardmapped version...")

r2 = shardmapped_f(y).block_until_ready()
print("Done")
print(jnp.allclose(r1.reshape(100 * n, 100, 100), r2))

Output:
JAX version: 0.4.23
[CpuDevice(id=0)]
number of devices:  1
Doing pmapped version...
Done
Doing shardmapped version...
Done
True

Attached the gists on CPU and GPU for reference.
I also check the code on Cloud vm with Ubuntu 22.04 having 4 CPUs and 4 GPUs and JAX version 0.4.24 and it worked fine.

Please verify with latest version of JAX and let us know if the issue still exists.
Thank you

Yes, it is now fixed. I dumped the buffer allocation file and both at the same and start with:
BufferAssignment:
allocation 0: size 4000000, parameter 0, shape |f32[100,100,100]| at ShapeIndex {}, output shape is |f32[100,100,100]|, maybe-live-out:
 value: <8 param.1 @0> (size=4000000,offset=0): f32[100,100,100]{2,1,0}
 value: <10 wrapped_slice @0> (size=40000,offset=0): f32[1,100,100]{2,1,0}
 value: <13 loop_slice_fusion @0> (size=4000000,offset=0): f32[100,100,100]{2,1,0}
allocation 1: size 4040128, preallocated-temp:
 value: <9 wrapped_copy @0> (size=4000000,offset=128): f32[100,100,100]{2,1,0}
 value: <11 collective-permute-start{} @0> (size=16,offset=0): (f32[1,100,100]{2,1,0}, f32[1,100,100]{2,1,0})
 value: <12 collective-permute-start{1} @0> (size=40000,offset=4000128): f32[1,100,100]{2,1,0}

Closing.

Is it possible this still doesn't work with TPU? https://github.com/google/jax/issues/15368#issuecomment-1963344697 @rajasekharporeddy. Also how can I produce a summary of the buffer assignment like above?

Hi @wbrenton
I tested it on colab TPU v2 with JAX version 0.4.26 and JAX nightly version 0.4.31.dev20240630 and it works fine.
Attaching gist for reference.
Thank you.


Tidying up some formatting

I loved your package, it's simple, elegant, and the only one I've found that works well in dart/flutter.
There were a few things I wanted to add to it, and I thought I'd open a PR in case you wanted to include any of them in the actual package.

I removed the requirement on quiver (again, not as clean, but less requirements on the outside world).
Added in lots of formatting for Strings. I needed to format a bunch of strings with special characters, xml code, escape characters, etc. So I've added in an extra function to handle all of these (and a list of all the special characters).
I appreciate the ability to sort the values if I want to, but often I didn't, so I made this optional.
Because I had a lot of long lines of text, I wanted to shorten them. I added a formatter that cuts them down to ~80 characters per line
The final return statement for toYamlString I'm not particularly proud of, but I haven't yet come up with a better way of moving the values next to the hyphen in lists instead of below.

As I said, it's not as elegant as your original package, but because I needed it to perform a little differently, I thought I'd check if you wanted to include any of them. Thanks!

hello @luuillu please approves this PR.

I needed to include this in one of my packages, so I published a fork. It's basically just the above changes. Feel free to look: https://pub.dev/packages/fhir_yaml

@Dokotela Thank you very much for your excellent idea contributed to this library.  the code already merged into master branch. I have published a new version package on pub.dev.

@luuillu Happy to have helped! Thanks for creating the package!


Add method chaining to all add* and remove* methods

This pull request add method chaining to all add* and remove* methods.
According to issue #625 this enables more compact code like
new Ajv().addSchema(mySchema).validate(schema, data)

I am still thinking about, how to get rid of the add ;)
What changes did you make?
I add return this to 6 methods: addSchema, addMetaSchema,  removeSchema, addFormat, addKeyword, removeKeyword.
updated jsdoc and the typescript definitions.
I added tests for the methods with an extra method test, except for addMetaSchema, because i couldn't find a good place for the test.
I hope i didn't oversee anything, if so let me know, would be great if it get merged

Thank you. API section in readme needs to be updated as well and there are also several examples that could use chaining: https://github.com/epoberezkin/ajv#api

Ok thx for your feedback, i will have a look and try to add something reasonable to the API doc.


Update use-the-add-azure-replica-wizard-sql-server.md

added warnings about this wizard only applicable for classic and not resource manager. also this wizard is not available in latest SSMS.

@ktoliver : Thanks for your contribution! The author(s) have been notified to review your proposed change.

Hi @MashaMSFT -- Could you review the proposed update to your article? Thanks!

#sign-off


[atom.sh] Remove REDIRECT_STDERR -h/-v flags

Identify the Bug
On Mac, the atom.sh (or /usr/local/bin/atom) won't output the full Atom help text when providing the --help or -h flags. This is due to STDERR being redirected to /dev/null for those flags.
Description of the Change
This solution simply removes the REDIRECT_STDERR=1 from those flags.
Alternate Designs
My first though was to separate the help flag from the version flag in the case block, and set ATOM_HELP=1
Then a little further down, simply run something like ${ATOM_APP}/Contents/MacOS/Atom -h, if ATOM_HELP is 1.
With a little bit of testing, I realised that the help text is output to STDERR, and I found my proposed change to be a much simpler one.
Possible Drawbacks
I am guessing that the STDERR was redirected for a reason for these flags. Which makes me think that they might output e.g. error messages, in some cases. But I have not been able to make Atom output anything other than the version/help text from the command with these flags.
Verification Process
I created a separate file for atom.sh outside of the Atom.app, and changed the lines directly after
if [ $OS == 'Mac' ]; then

where it checks if the script is a link, and sets the SCRIPT variable, to look like this:
if [ $OS == 'Mac' ]; then
  SCRIPT="$(readlink "/usr/local/bin/atom")"

After changing this, I could test that my changes to atom.sh worked as expected.
Release Notes

Fixed the issue where the Atom CLI tool won't output full help text.

From what I can tell, the test errors in Windows Powershell seems to be unrelated to my changes.

@artheus There has been no response to my comment above. Am closing this PR for now, but don't hesitate to reach out to me when you get time to address the comment. I will be happy to reopen this PR.


Want to know the range of unique of itemid

Thank you for making detailed document.
I understand item id is the unique identifier of the Office.context.mailbox.item, but I'm not sure about the range of unique from this document. Is this id unique all over the cloud service (Office 365 and Outlook.com) and all existing on-premises Exchange Servers? Or is the id unique only in cloud service (O365 and Outlook.com)?

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 32e25ec7-e2ef-2bbc-4c35-9edda710d352
Version Independent ID: 03a0d879-c192-490a-3086-de7a3f6ad542
Content: Office.context.mailbox.item - requirement set 1.4 - Office Add-ins
Content Source: docs/reference/objectmodel/requirement-set-1.4/office.context.mailbox.item.md
Product: outlook
Technology: add-ins
GitHub Login: @o365devx
Microsoft Alias: o365devx

Thanks for the question @NT-D.
@exextoc, could provide an answer?

Assigning @ElizabethSamuel-MSFT to follow up on any necessary documentation changes.

@NT-D is there a specific scenario you are referring to?

@exextoc  Sorry for my late reply. Let me describe scenario.
In our app backend, we need unique id for db record, and need to have relation with each event in Office 365. Because our end-users exists across tenants, I want to know the uniqueness.

FYI: This document written by Identity team is useful example. It describe oid is GUID and range of uniqueness well.
https://docs.microsoft.com/en-us/azure/active-directory/develop/id-tokens
-- | -- | --

oid
String, a GUID
The immutable identifier for an object in the Microsoft identity system, in this case, a user account. This ID uniquely identifies the user across applications - two different applications signing in the same user will receive the same value in the oid claim. The Microsoft Graph will return this ID as the id property for a given user account. Because the oid allows multiple apps to correlate users, the profile scope is required to receive this claim. Note that if a single user exists in multiple tenants, the user will contain a different object ID in each tenant - they're considered different accounts, even though the user logs into each account with the same credentials. The oid claim is a GUID and cannot be reused.

The item id returned by the API is the EWS item id of the item. This is the primary documentation for the EWS item id: https://docs.microsoft.com/en-us/exchange/client-developer/exchange-web-services/ews-identifiers-in-exchange
Global uniqueness of the id isn't guaranteed. The documentation says only this about how the ID is constructed:

Identifiers in Exchange are opaque. For example, the EwsId is created from several pieces of information that are not important to you as the developer, but are important to Exchange.

Since it's not guaranteed to be globally unique, it's not safe to use as the only identifier within a database used by multiple organizations.
Since you are storing the id, please be aware that the item id changes when the item is moved between folders:

Don't assume that your ID will always be valid if you need to retrieve the item at a later time. If an item is moved in the store, the ID can change because of the way a move is handled. An item is actually copied, and a new ID is generated, and then the original item is deleted.

Also, please be aware if your add-in supports shared folder scenarios now or in the future, items shared with multiple users may also share an identifier (because multiple users have access to the same item). Then, it depends on your application whether you want multiple users accessing the same item to share information in your database as well.

Glad to help, and thanks for the feedback.
@ElizabethSamuel-MSFT for feedback on adding a link to EWS Identifiers from the itemId documentation.
If you do need an immutable ID, this REST API can exchange a REST id for an immutable id. (The EWS id can be converted locally to the URL-safe REST id format).


EL-1106 MTR phase 1 cleanup

Jira ticket
remove mtr flags and references and update URL for PDF and guidance links
Guidance to review

Checklist
Before you ask people to review this PR:

Tests and rubocop should be passing
Branch is generally up to date with main Github - definitely no conflicts
No unnecessary whitespace changes. These make diffs harder to read and conflicts more likely.
PR description says what changed and why, with a link to the JIRA story.
Diff has been checked for unexpected changes being included.
Commit messages say why the change was made.

Hey @willc-work, I think this PR should be re-named to EL-1105?


query-frontend logs failed to parse host port for http log decision

Thanos, Prometheus and Golang version used:
Version	0.21.1
Revision	3558f4a5adc8d4068f82528570a439d440f98b9d
Branch	HEAD
BuildUser	circleci@eb60c35e294a
BuildDate	20210604-12:24:45
GoVersion	go1.16.4

What happened:
I have deployed the query-frontend to the Kubernetes and there are a lot of logs like this in the container:
level=error ts=2021-06-09T04:29:40.506819592Z caller=http.go:48 protocol=http http.component=server msg="failed to parse host port for http log decision" err="address thanos.apps.x.y.z: missing port in address"

Here are the command-line flags for the query-frontend:
            - --query-frontend.compress-responses
            - --query-frontend.downstream-url=http://thanos-query-http.thanos.svc:10902
            - --query-frontend.log-queries-longer-than=5s
            - --query-range.split-interval=24h
            - --query-range.response-cache-max-freshness=1m
            - --query-range.max-retries-per-request=5
            - --query-range.response-cache-config-file=/etc/thanos/frontend.yml
            - --labels.max-retries-per-request=10
            - --labels.split-interval=12h
            - --cache-compression-type=snappy

What you expected to happen:
I haven't defined any address for query-frontend in the command-line arguments, if the logger is fetching the address from the user's request why it cannot determine the port address?
How to reproduce it (as minimally and precisely as possible):
Full logs to relevant components:
level=error ts=2021-06-09T04:29:40.506819592Z caller=http.go:48 protocol=http http.component=server msg="failed to parse host port for http log decision" err="address thanos.apps.x.y.z: missing port in address"

Anything else we need to know:

I feel this issue is related to the logging middleware. CC @yashrsharma44.
The error happens at https://github.com/thanos-io/thanos/blob/main/pkg/logging/http.go#L43-L51. If there is no port in the address, can we do something to skip this error?

@yeya24  Hi. I also encountered this error log, although it does not affect the use of the service. I want to introduce the situation on my side.
This is an example of my current architecture

This is my version information and error log
thanos --version
thanos, version 0.21.0 (branch: HEAD, revision: d19aea21b178a6c08838c6a05b834dd44ef14273)
  build user:       root@30b80273f41d
  build date:       20210603-13:49:01
  go version:       go1.16.4
  platform:         linux/amd64

{"caller":"http.go:48","err":"address thanos.demo.io: missing port in address","http.component":"server","level":"error","msg":"failed to parse host port for http log decision","protocol":"http","ts":"2021-06-11T17:55:11.136477708Z"}

I am also seeing the same issue on a few Thanos clusters I am using. 0.19.0 did not have this issue, but when I upgraded to 0.21.1 I started seeing lots of these logs.

This problem does not happen on  standalone deployment as the r.Host maps to --http-address which contains a port #.

If app. like Query-frontend is deployed under a hostname which in turn mapped to --http-address (host + port) of query-frontend component (eg: route on a Kube. cluster), then this problem shows up as SplitHostPort() of net package expect a ':' and throw up an error.

Apart from lots of parsing errors in the log file, this problem results in 'not applying the decider logic' for logging.

This PR fixes this issue.

These might be related:
https://github.com/thanos-io/thanos/issues/4139
https://github.com/thanos-io/thanos/issues/4118
https://github.com/thanos-io/thanos/issues/4334
Linking them for reference

Should be solved in #4348. Thanks @spaparaju


ModuleNotFoundError: No module named 'django_extensions'

i follow Setup for development , but after run docker show this error.
==> 08:51:30 ==> Collecting statics...
==> 08:51:30 ==> Send via Slack info about service version and network
==> 08:51:30 ==> Running Gunicorn...
[2022-08-18 08:51:30 +0000] [1] [INFO] Starting gunicorn 20.1.0
[2022-08-18 08:51:30 +0000] [1] [INFO] Listening at: unix:/nginx/gunicorn.socket,http://<IP_ADDRESS>:8888 (1)
[2022-08-18 08:51:30 +0000] [1] [INFO] Using worker: gevent
[2022-08-18 08:51:30 +0000] [13] [INFO] Booting worker with pid: 13
[2022-08-18 08:51:30 +0000] [13] [INFO] Making Psycopg2 Green
[2022-08-18 08:51:30 +0000] [13] [INFO] Made Psycopg2 Green
Traceback (most recent call last):
  File "/app/manage.py", line 30, in <module>
    execute_from_command_line(sys.argv)
  File "/usr/local/lib/python3.10/site-packages/django/core/management/__init__.py", line 446, in execute_from_command_line
    utility.execute()
  File "/usr/local/lib/python3.10/site-packages/django/core/management/__init__.py", line 420, in execute
    django.setup()
  File "/usr/local/lib/python3.10/site-packages/django/__init__.py", line 24, in setup
    apps.populate(settings.INSTALLED_APPS)
  File "/usr/local/lib/python3.10/site-packages/django/apps/registry.py", line 91, in populate
    app_config = AppConfig.create(entry)
  File "/usr/local/lib/python3.10/site-packages/django/apps/config.py", line 228, in create
    import_module(entry)
  File "/usr/local/lib/python3.10/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1004, in _find_and_load_unlocked
ModuleNotFoundError: No module named 'django_extensions'
[2022-08-18 08:51:31 +0000] [13] [ERROR] Exception in worker process
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/gunicorn/arbiter.py", line 589, in spawn_worker
    worker.init_process()
  File "/usr/local/lib/python3.10/site-packages/gunicorn/workers/ggevent.py", line 146, in init_process
    super().init_process()
  File "/usr/local/lib/python3.10/site-packages/gunicorn/workers/base.py", line 134, in init_process
    self.load_wsgi()
  File "/usr/local/lib/python3.10/site-packages/gunicorn/workers/base.py", line 146, in load_wsgi
    self.wsgi = self.app.wsgi()
  File "/usr/local/lib/python3.10/site-packages/gunicorn/app/base.py", line 67, in wsgi
    self.callable = self.load()
  File "/usr/local/lib/python3.10/site-packages/gunicorn/app/wsgiapp.py", line 58, in load
    return self.load_wsgiapp()
  File "/usr/local/lib/python3.10/site-packages/gunicorn/app/wsgiapp.py", line 48, in load_wsgiapp
    return util.import_app(self.app_uri)
  File "/usr/local/lib/python3.10/site-packages/gunicorn/util.py", line 359, in import_app
    mod = importlib.import_module(module)
  File "/usr/local/lib/python3.10/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 883, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/app/config/wsgi.py", line 38, in <module>
    application = get_wsgi_application()
  File "/usr/local/lib/python3.10/site-packages/django/core/wsgi.py", line 12, in get_wsgi_application
    django.setup(set_prefix=False)
  File "/usr/local/lib/python3.10/site-packages/django/__init__.py", line 24, in setup
    apps.populate(settings.INSTALLED_APPS)
  File "/usr/local/lib/python3.10/site-packages/django/apps/registry.py", line 91, in populate
    app_config = AppConfig.create(entry)
  File "/usr/local/lib/python3.10/site-packages/django/apps/config.py", line 228, in create
    import_module(entry)
  File "/usr/local/lib/python3.10/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1004, in _find_and_load_unlocked
ModuleNotFoundError: No module named 'django_extensions'
[2022-08-18 08:51:31 +0000] [13] [INFO] Worker exiting (pid: 13)
[2022-08-18 08:51:31 +0000] [1] [INFO] Shutting down: Master
[2022-08-18 08:51:31 +0000] [1] [INFO] Reason: Worker failed to boot.

 1057  source venv/bin/activate
 1058  pip3 install -r requirements-dev.txt
 1059  pre-commit install -f
 1060  ./run_tests.sh
 1061  docker-compose -f docker-compose.yml -f docker-compose.dev.yml up

Ok.
For now, you can remove in config/settings/local.py django_extensions
I mean change this line INSTALLED_APPS += ["debug_toolbar", "django_extensions"]
For this INSTALLED_APPS += ["debug_toolbar"]
Remember rebuild web:
docker-compose -f docker-compose.yml -f docker-compose.dev.yml build web
This is a temporary solution if docker is for development then I think that should install also requirements.dev.txt

@moisses89 thank you so much. it works.

@Uxio0 we should take a look to this, basically the issue is that docker is not installing requirements.dev.txt but is using local configuration that includes django_extensions should we create separate dockerfile for dev and other for prod?

No, not at all. I think we should include django-extensions on requirements.txt, as the extensions are so useful even for our production environments


Add metrics about grpc async client and server connections

What problem does this PR solve?
Issue Number: ref #6225
Problem Summary:
What is changed and how it works?
Check List
Tests

[ ] Unit test
[ ] Integration test
[x] Manual test (add detailed scripts or steps below)
[ ] No code

You can see these two metric in tiflash_summary/Threads_CPU here

Side effects

[ ] Performance regression: Consumes more CPU
[ ] Performance regression: Consumes more Memory
[ ] Breaking backward compatibility

Documentation

[ ] Affects user behaviors
[ ] Contains syntax changes
[ ] Contains variable changes
[ ] Contains experimental features
[ ] Changes MySQL compatibility

Release note

None

/cc @LittleFall @windtalker

can you add some metric screenshots in pr description? we may check it in the future.

add a "limit" line like other metrics in this section?

done

/merge

/run-all-tests


[uart/dv] end tx item earilier to avoid accumulating freq diff

Use only 0.9 period for the last cycle as design and agent aren't
using same clock. If agent freq is slower and design continues
sending tx item, the freq diff will be accumulated.
Ending current item earlier to allow agent to re-establish clock
to sample next item
another 2 minor updates

use realtime as period contains decimal value
clean up unused code

Signed-off-by: Weicai Yang<EMAIL_ADDRESS>

@eunchan
before we used fixed period to sample TX data but design and agent aren't using same clock. expect each transaction lasts 10 uart cycles (assume no parity)
If agent freq is slower and design continues sending tx item, the freq diff will be accumulated. Then, will sample in a wrong position.
So, I shorten the last cycle to 0.9 period, so it ends current item earlier to allow agent to re-establish clock for next item. I think the diff should be less than 0.1 period for an item.
For the RX, will keep the period not change. driver may be slower freq than design. but design should be able to adjust that diff.
WDYT?

LGTM!

OK. So this is due to the UART rx agent doesn't detect the START bit to self-centering the latch point, am I right?
As mentioned in the spec, NCO can have 2.5% offset (if peripheral clock is 100% correct). Meaning that at slow baudrate, 90% of last cycle may not be enough..

OK. So this is due to the UART rx agent doesn't detect the START bit to self-centering the latch point, am I right?

@eunchan
RX agent starts to monitor the START bit when the previous one is completed. Once it sees the START bit, will start the uart clock and the center of the clock (0.5 period) is the latch point. If previous one is completed late, the latch point will be shifted as well. If design keeps sending data, it would shift further and further.

As mentioned in the spec, NCO can have 2.5% offset (if peripheral clock is 100% correct). Meaning that at slow baudrate, 90% of last cycle may not be enough..

I will reduce the last cycle to 0.7 period, so it can monitor START bit before it's sent from design. WDYT?

On 8/25/20 1:41 PM, weicaiyang wrote:

OK. So this is due to the UART rx agent doesn't detect the START
bit to self-centering the latch point, am I right?

@eunchan https://github.com/eunchan
RX agent starts to monitor the START bit when the previous one is
completed. Once it sees the START bit, will start the uart clock and
the center of the clock (0.5 period) is the latch point. If previous
one is completed late, the latch point will be shifted as well. If
design keeps sending data, it would shift further and further.

Maybe I didn't understand the issue exactly in previous comment ( and
maybe here too). "If previous one is completed late" means that the DUT
sends the UART Tx slightly faster than desired baudrate so that the VIP
sees start bit not from the edge but later. Does it describe the issue
correctly?

As mentioned in the spec, NCO can have 2.5% offset (if peripheral
clock is 100% correct). Meaning that at slow baudrate, 90% of last
cycle may not be enough..

I will reduce the last cycle to 0.7 period, so it can monitor START
bit before it's sent from design. WDYT?

So the design determines permissible error rate in order not to shifted
by more than half a baudrate in a byte transfer. So theoretically the
center point  of the last byte also can be the last point of the valid
data (not sure exactly checked this so may be wrong).
I think checking START bit right after the last beat center line is the
best.

Maybe I didn't understand the issue exactly in previous comment ( and maybe here too). "If previous one is completed late" means that the DUT sends the UART Tx slightly faster than desired baudrate so that the VIP sees start bit not from the edge but later. Does it describe the issue correctly?

yes, VIP uses ideal clock. if uart period is 100ns, it waits for 100ns * 10 (assume no parity) before start to monitor the next START bit. The next item could start after the edge of START bit. Sorry for the confusing.

So the design determines permissible error rate in order not to shifted by more than half a baudrate in a byte transfer. So theoretically the center point  of the last byte also can be the last point of the valid data (not sure exactly checked this so may be wrong). I think checking START bit right after the last beat center line is the best.

sounds good. will change VIP to end the transaction when it samples the STOP bit (center of last cycle), then start to monitor the next START bit

Not entirely happy about this fix, since it does not transmit all bits at equal intervals (a real UART would never behave this way).
A better, more realistic fix would be to add an internal clock within uart_if. The said clock would run at 24MHz (will be configurable), and the tick pulses will be generated by dividing it. Then the agent's baud rate would also be approximate, bridging the gap with the design. A config knob can be added to choose between "perfect" baud rate and this approx baud rate.
I am fine with this landing as a quick workaround with the goal of fixing it properly later.

Not entirely happy about this fix, since it does not transmit all bits at equal intervals (a real UART would never behave this way).
A better, more realistic fix would be to add an internal clock within uart_if. The said clock would run at 24MHz (will be configurable), and the tick pulses will be generated by dividing it. Then the agent's baud rate would also be approximate, bridging the gap with the design. A config knob can be added to choose between "perfect" baud rate and this approx baud rate.
I am fine with this landing as a quick workaround with the goal of fixing it properly later.

Do you mean to create another clock which behaves close to design's clock?
I feel uart transfers data without providing a clock, which means certain freq diff between 2 components is expected. Uart spec defines last bit Stop is same as idle, which could be used by receiver to re-establish the clock. It makes sense that receiver considers the item is completed once it latches the STOP bit, and then start to look for the edge of START bit for the next item. This helps to adjust the freq diff as long as it's less than 0.5 clock period for a transaction. When agent drives the TX, it's using ideal freq and our design as receiver can handle this freq diff.
we can discuss more in DV meeting.

@eunchan I'm thinking to test design clock should not shift more than 1/4 cycle comparing to a perfect clock. If our design attaches to another device, that device could have same amount of clock offset but off in the other direction. And the total offset should be less than half a baud rate.
So, I think I can have the driver to check RX data is stable from 1/4 period to 3/4 period for each cycle, based on the perfect clock. Does it make sense?

@weicaiyang can this one be merged now?

@weicaiyang can this one be merged now?

yes, but I need someone approve it. Will also discuss with Sri if there is other better solution


Send tracking

Describe your changes

Add classifier to Send (which adds rawgrade and type to doc similar to Route)
Add bestSends and totalSends maps to User object and document
Update these when sending a route
add getBestSendClassifier and getTotalSends, both of which take the query route type

Issue ticket number and link
closes #131
closes #132

I also had to fix getUserByUsername on this to test. Should have split the commit but here we are :shrug:

I also had to fix getUserByUsername on this to test. Should have split the commit but here we are 🤷

🥳 -- we're about to use this API on client


Add Slf4jMdcDataInvocationWrapper unit tests

References GUS-W-9123596

Codecov Report

Merging #87 (dab515c) into main (8eb4c54) will increase coverage by 1.39%.
The diff coverage is n/a.

@@             Coverage Diff              @@
##               main      #87      +/-   ##
============================================
+ Coverage     63.89%   65.29%   +1.39%
- Complexity      435      440       +5
============================================
  Files            91       91
  Lines          1576     1576
  Branches        101      101
============================================
+ Hits           1007     1029      +22
+ Misses          559      537      -22
  Partials         10       10

Impacted Files
Coverage Δ
Complexity Δ

...sfjavafunction/Slf4j1MdcDataInvocationWrapper.java
73.33% <0.00%> (+73.33%)
5.00% <0.00%> (+5.00%)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 8eb4c54...dab515c. Read the comment docs.


Too many args is suppressed for records but not tables

The codegenerator generates a cilppy allow for too many args for a record ctor but not for a table ctor

Fixed by #281


[ALLUXIO-2602]Make RenameOptionsTest final to avoid accidental subclass extension.

https://alluxio.atlassian.net/browse/ALLUXIO-2602
Make RenameOptionsTest final to avoid accidental subclass extension.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Alluxio-Pull-Request-Builder/14023/
Test PASSed.


Tidy up DEV_GUIDE shell examples and document additional filters profile

We've got a bit of inconsistency with placeholders in the text.  We've got curly brackets here and dollars used down below $your_quay_username$ etc.  We should use a single style.  I don't care which.

Originally posted by @k-wall in https://github.com/kroxylicious/kroxylicious/pull/542#discussion_r1293108667

Also we should document the withAdditionalFilters profile https://github.com/kroxylicious/kroxylicious/pull/542#issuecomment-1676907307

We've got a bit of inconsistency with placeholders in the text. We've got curly brackets here and dollars used down below $your_quay_username$ etc. We should use a single style. I don't care which.

My vote is for something that actually works for variable substitution in the terminal, i.e. $My_Awesome_Variable. That way if I want to I can set $My_Awesome_Variable to whatever my substitute value is in my terminal, then paste in the command as-is and not have to edit it to make it work.

Triaged 16/08/23


EventBaseConfig::FEATURE_FDS not supported on Windows

If I use https://pecl.php.net/package/event library on Windows then I got following error out of the box
PHP Warning:  EventBase::__construct(): event_base_new_with_config: no event mechanism available

After small investigation I found that if following line commented then all works as expected
https://github.com/reactphp/event-loop/blob/master/src/ExtEventLoop.php#L47
$config->requireFeatures(EventBaseConfig::FEATURE_FDS);

Examples
Server: https://github.com/inri13666/web-socket-echoserver
Client: https://github.com/inri13666/web-socket-test
Library Info
composer show | grep loop
react/event-loop                     v1.1.0  ReactPHP's core reactor event loop that libraries can use for evented I/O.

Environment
systeminfo | findstr /B /C:"OS Version" /C:"System Type" /C:"System Locale"

OS Version:
10.0.17763 N/A Build 17763

System Type:
x64-based PC

System Locale:
en-us;English (United States)

php -i | findstr /B /C:"PHP Version" /C:"Architecture" /C:"Compiler" /C:"Configure Command" /C:"Zend Extension Build"

PHP Version
7.1.26

Compiler
MSVC14 (Visual C++ 2015)

Architecture
x64

Zend Extension Build
API320160303,TS,VC14

Configure Command:  cscript /nologo configure.js  "--enable-snapshot-build" "--enable-debug-pack" "--with-pdo-oci=c:\php-snap-build\deps_aux\oracle\x64\instantclient_12_1\sdk,shared" "--with-oci8-12c=c:\php-snap-build\deps_aux\oracle\x64\instantclient_12_1\sdk,shared" "--enable-object-out-dir=../obj/" "--enable-com-dotnet=shared" "--with-mcrypt=static" "--without-analyzer" "--with-pgo"
PHP Extensions
php -m
[PHP Modules]
bcmath
bz2
calendar
com_dotnet
Core
ctype
curl
date
dom
event <====
exif
fileinfo
filter
gd
hash
iconv
intl
json
ldap
libxml
mbstring
mcrypt
memcache
mysqli
mysqlnd
openssl
pcre
PDO
pdo_mysql
pdo_pgsql
pdo_sqlite
pdo_sqlsrv
pgsql
Phar
readline
Reflection
session
SimpleXML
soap
sockets
SPL
sqlsrv
standard
tokenizer
wddx
xdebug
xml
xmlreader
xmlrpc
xmlwriter
xsl
zip
zlib

[Zend Modules]
Xdebug

@inri13666 Thanks for reporting and providing the platform info, it looks like this might in fact be a bug on Windows with ext-event.
Can you verify this by running the test suite and showing its output prior to your change and with your change applied? If this fixes the problem, a simple PR with your changes would be much appreciated! :shipit:

Actual master phpunit results on Windows platform
phpunit-master.txt
If I commented out mentioned line
phpunit-patch.txt
Later I'll prepare PR for this error
EventBase::__construct(): event_base_new_with_config: no event mechanism available
But right now I'm not sure how to fix on windows
1) React\Tests\EventLoop\Timer\ExtEventTimerTest::testAddPeriodicTimerCancelsItself
Failed asserting that 0.0043990612030029297 is equal to 0.0050000000000000001 or is greater than 0.0050000000000000001.

FEATURE_FDS allows file descriptors as well as sockets to be watched, removing this feature is a BC. The factory should probably just skip the ext-event loop if the OS is Windows.

@inri13666 Thank you for your patience, I've tried to reproduce this locally but have been unable to get ext-event running on Windows so far. Can you provide some instructions on how to best reproduce your setup on a clean Windows 10 installation so I can look into providing an automated test setup for this project?
The unrelated build error regarding timers is going to be fixed via #203.

The DLL can be downloaded here https://pecl.php.net/package/event
I have tested it with following extension version
event
Event support => enabled
Sockets support => enabled
Debug support => disabled
Extra functionality support including HTTP, DNS, and RPC => enabled
OpenSSL support => disabled
Thread safety support => enabled
Extension version => 2.4.2
libevent2 headers version => 2.0.22-stable

PHP version
PHP Version => 7.1.26

System => Windows NT DESKTOP-13666 10.0 build 17763 (Windows 10) AMD64
Build Date => Jan  9 2019 21:47:23
Compiler => MSVC14 (Visual C++ 2015)
Architecture => x64
Configure Command => cscript /nologo configure.js  "--enable-snapshot-build" "--enable-debug-pack" "--with-pdo-oci=c:\php-snap-build\deps_aux\oracle\x64\instantclient_12_1\sdk,shared" "--with-oci8-12c=c:\php-snap-build\deps_aux\oracle\x64\instantclient_12_1\sdk,shared" "--enable-object-out-dir=../obj/" "--enable-com-dotnet=shared" "--with-mcrypt=static" "--without-analyzer" "--with-pgo"
Server API => Command Line Interface
Virtual Directory Support => enabled
Configuration File (php.ini) Path => C:\Windows
Scan this dir for additional .ini files => (none)
Additional .ini files parsed => (none)
PHP API => 20160303
PHP Extension => 20160303
Zend Extension => 320160303
Zend Extension Build => API320160303,TS,VC14
PHP Extension Build => API20160303,TS,VC14

@inri13666 Thank you, I've been able to reproduce this locally with the help of @sitedyno (ext-event needs to be loaded after ext-sockets) and have prepared a PR to add an automated test on Windows and fix this issue. Expect this PR soon, this is currently waiting for #203/#204 first :+1:


chore(example): layout improvements

fixes an issue where the next and previous buttons jump because the width of the play / pause /loading indicator changes when switching between playing, pausing & buffering.
show ‘Live Stream’ text instead of non-functioning progress bar when track has isLiveStream: true
debounce showing of buffering indicator by 250ms to avoid showing too quickly (and too briefly)

(something odd going on - don't merge this)

So odd - https://github.com/doublesymmetry/react-native-track-player/pull/1703/files is showing changes in src/hooks.ts – but those changes are already in main branch.. a little confused by this...

@puckey I think I know what's going on. Did you branch off your the same branch you made those changes in?
If that's the case the issue is because we're currently using GitHubs "Squash and merge" functionality. Which means that the commit SHA in main is different then the one in your branch. You're either going to have to cherry-pick into a new branch off of main re-apply your changes on a new branch off of main.

@puckey I think I know what's going on. Did you branch off your the same branch you made those changes in?
If that's the case the issue is because we're currently using GitHubs "Squash and merge" functionality. Which means that the commit SHA in main is different then the one in your branch. You're either going to have to cherry-pick into a new branch off of main re-apply your changes on a new branch off of main.

Thanks yeah this is the issue exactly - fixing now

Should be good to go now


Hide "Code Preview Bar"

Hi, is there an setting or much better menu entry to disable/toggle this annoying "Code Preview Bar" on/off beside the right vertical ScrollBar?

you want it like this?

It's Ctrl + B

Are you talking about the minimap?
If so, you can disable it in the preferences, using this command: "editor.minimap.enabled": false,

@fbnlsr
Yes I meant the minimap. Thanks.

Hello, exist how make will list functions of the .js file?

How can i remove the left side? i mean the line number. thansk.


DCOS_OSS-1238: fix(externalVolumes): Remove size if docker

This commit will introduce that only external
volumes in the MESOS container type are allowed to
have a size.
Closes DCOS_OSS-1238
Checklist

[x] Did you add a JIRA issue in a commit message or as part of the branch name?
[ ] Did you add new unit tests?
[ ] Did you add new integration tests?
[ ] If this is a regression, did you write a test to catch this in the future?

Let's adjust the tooltip wording to align with https://mesosphere.github.io/marathon/docs/external-volumes.html and maybe even put the link in there.

Does
Docker Runtime only supports the default size for implicit volumes, please select Mesos Runtime if you want to modify the size.

Sound better to you?

I checked it out, and worked as expected for my fresh eyes. I agree with all of @nLight 's comments. 👍

LGTM! 🚢


fix formatting for configure blobstore for gcp section

There is a weird section in the docs:
https://bosh.io/docs/director-configure-blobstore/#properties-blobstore-provider-gcs-credentials_source-static-json_key-director-blobstore-service-account-file-bucket_name-test-bosh-bucket-storage_class-regional-agent-blobstore-credentials_source-static-json_key-agent-service-account-blobstore-file
also the yaml snippet appears bigger than for the other ones. Maybe this space helps?

I checked with locally rendered docs that this helps.


Prevent multiple connections from running migrations at the same time

Adds locking when migrations are started
forceFreeMigrationsLock function to allow removing the lock manually

@rhys-vdw could you give this the final review. @dustinmartin made this almost ready, but didn't have time to give it a last touch in #1067.

@elhigu Yep, will review this evening.

@elhigu Looking good.

@elhigu Thanks or finishing this up. Very much appreciated.
When do you think the next release of knex will be that will include these changes?

@dustinmartin Should do one soon. @elhigu is there anything else you'd like to see in the next release?

@rhys-vdw there is still that one pull request about sqlite3 dialect escaping which I would like to get in. It fixes also some regressions from postgres string literal escaping from getting to release.


After change password ownership on connection is lost.

@shamis I found the issue is on the server side so to avoid conflicts with your saml changes will assign it to you to fix it there.
The problem is that the user_id is not set when we get error ?PasswordChangeNeeded, I was able to fix this issue by adding this 2 lines on dderl_session 206
{[UserId],true} = imem_meta:select(ddAccount, [{#ddAccount{name=State#state.user, id='$1',_='_'}, [], ['$1']}]),
State1 = State#state{user_id = UserId},


rdp fully supported but not working

already uninstall and reinstall all but no working this is my problem.

help me guys ;C

Not RDP Wrapper problem, contact Telegram support chat.


Inconsistent inference in 2.6.1

TypeScript Version:  2.6.1 / 2.7.0-dev.20171110
strict: true
strictFunctionType: false

Code
type List<T> = ArrayLike<T> | T[];

declare function pipe<T1, T2, R>(
  f1: (x1: T1) => T2,
  f2: (x2: T2) => R
): (x: T1) => R;

type R1<U> = U[];
type R2<U, K extends string> = U[] | Record<K, U>;

type F1<T, U> = {
  (list: List<T>): R1<U>;
  <K extends string>(target: List<T> | Record<K, T>): R2<U, K>;
};

type F1_1<T, U> = {
  (list: List<T>): R1<U>;
};

type F1_2<T, U> = {
  <K extends string>(target: List<T> | Record<K, T>): R2<U, K>;
};

declare const fn1: F1<string, number>;
declare const fn1_1: F1_1<string, number>;
declare const fn1_2: F1_2<string, number>;
declare const fn2: (x: List<any>) => any;

pipe(fn1, fn2);
//        ^^^ error in v2.6.1
// Argument of type '(x: List<any>) => any' is not assignable to parameter of type '(x2: R2<number, any>) => any'.
// Types of parameters 'x' and 'x2' are incompatible.
//   Type 'R2<number, any>' is not assignable to type 'List<any>'.
//     Type 'Record<any, number>' is not assignable to type 'List<any>'.
//       Type 'Record<any, number>' is not assignable to type 'ArrayLike<any>'.
//         Property 'length' is missing in type 'Record<any, number>'.

pipe(fn1_1, fn2); // no error
pipe(fn1_2, fn2); // no error

Expected behavior:

All passed
or pipe(fn1, fn2) and at least one of pipe(fn1_1, fn2) and pipe(fn1_2, fn2) throw error

If pipe(fn1, fn2) should throw error, at least one of pipe(fn1_1, fn2) and pipe(fn1_2, fn2) should also throw error since F1 = F1_1 + F1_2.
Actual behavior:
Only pipe(fn1, fn2) throw error.
NOTE: it was all passed before v2.6.1

Original post: https://github.com/types/npm-ramda/issues/311

Hello! Long time no response~ So I looked at this today and the error is one I'd categorize as a "correct" one. fn1_1's overloads indicate it can return either a {}[] or a {}[] | Record<string, {}>, which is, ofc, just {}[] | Record<string, {}>. fn2 cannot take a Record<string, {}> as a parameter, hence the error. pipe(fn1_2, fn2) should also be an error for the same reason (IMO). It's not, AFAIK, because we replace  K with never (instead of {}) because we find it in an argument position instead of a return position first. Because of that, Record<never, string> looks a lot like just never and it happens to typecheck. Looks like this happens because instantiateTypeWithSingleGenericCallSignature attempts to instantiate ArrayLike<string> | string[] | Record<K, string> in the context of {} and decides that K as never is most appropriate (which, in an isolated bit of reasoning, it is). The overloads don't work the same way because we don't use instantiateTypeWithSingleGenericCallSignature and go the long way around instead (and get a different, more correct, answer).

Checked at latest and there are now errors on the first and third calls; sounds like the middle one is hitting a limitation.


Region Playlist font size?

Hello.  Being able to increase the size of the text in the Region Playlist would greatly improve readability in a live performance.  I know you can switch to the "locked" view but alot of the functionality goes out the window in that mode.  Is there a way I can increase the size of the text in the backend somewhere?  Thanks!  My band lives and dies by the region playlist!
Also, I use the Notes view to view currently selected region but I need to be able to see what the next song is quickly without squinting.
Thanks again!

Is there any update on this, any related feature or any workaround?


piping final output to 'export-csv' gives undesired results

@IISResetMe Hey, thanks for this great module, and I really enjoyed watching the Southhampton User Group call. Keep up all the good work.
I have some 500 - 800 line scripts that currently run from anywhere between 3 to 6 hours if they complete at all. And for those bigger scripts, I think it would be really neat if I could relatively easily 'preserve' the output of the PSProfiler so I can keep coming back to it even days later and look at other sections to be improved upon.
But maybe I was/am trying to do this the 'wrong' way? Whcih more than possible, even likley actually. Anyway if I run something like:
Measure-Script -Path MyScript.ps1 | Tee-Object -Variable Measurements | Export-csv MyScript_psprofiler_output.csv
then the output is:
"LineNo","ExecutionTime","TimeLine","Line","SourceScript"
"1","00:00:00","TimeLine","# test-psprofiler-1","C:temp\test-psprofiler-1.ps1"

Which is not all that useful for what I wanted.
But what I would like to get out is, more something like:
"Count"," Line","Time Taken","Statement"
5757,87,00:01.3884714,"$strdate = (Get-Date).ToString("dd MMMM yyyy")"

Am I doing it the 'wrong' way or can I 'massage' the output such that I get what I was looking for, or is it something you would have to add as a new feature in PSProfiler? And is it something you think others might find helpful to?
Or maybe I just need to learn how to 'preserve' or save the contents of the array $Measurements that I did create.
Hope this makes sense. Either way thanks again for this great module.

Hey @TheBigBear!
You're not doing anything wrong, but you will need to "massage" the output a little bit to replicate the default format view:
Measure-Script {
  Start-Sleep -Seconds 2
} |Select @{Name='Count';Expression={$_.TimeLine.GetCount()}},@{Name='Line';Expression='LineNo'},@{Name='Time Taken';Expression={$_.TimeLine.GetTotal()}},@{Name='Statement';Expression='Line'}

The reason PSProfiler defers this step to the formatting subsystems is simple: preservation of underlying data points - allowing you to do more detailed analysis, useful for plotting the runtime complexity characteristics of the individual statement:
Measure-Script {
  $array = @()
  0..999 |ForEach-Object {
    $array += $_
  }
} |Select * -ExpandProperty TimeLine|Select * -ExpandProperty TimeSpans |Select LineNo,Ticks |Export-Csv .\detailedReport.csv

Hope this helps!


Little rework of existing yaml flatpacks

some id changing and yaml cleaning

all migration in place

AmePartFlatpackStealObjective is missing.

Also CI fail is real.


Define the repository setup

The TIKI SDK iOS repository must be set up for receiving open-source contributors.
I am listing some changes here for discussion. It will be implemented after defining those with the team.

The main branch is locked for being pushed. Just receive PRs.
To keep a linear branch history, just the main branch accepts PRs.
For each PR, a new branch must be created with the following pattern: [type]/[subject]

feat: new feature (minor release)
breaking: add a feature with breaking changes (major release)
fix: fix a bug
release

All PRs trigger the "Test" action that will run the repository tests. It is required that all tests pass to merge the PR.
Accepting a PR in the main branch from release branch triggers the "Release" action that will:

run tests
change the debug version of the Packages to the release version and commit
create a new tag
create a new release
change the Packages version back to develop and commit
update the docs
commits in the main branch with the title "[version] develop" and push

Create an action that checks the dependencies and creates PRs to update it. Dependabot does not work with SPM.
Add static code analysis

This setup will be used as the base for the other SDK repositories.


[APINotes] Add Swift 3 SwiftName for NSXMLDTDKind

In the SDK shipped with Swift 3, NSXMLNodeKind's enumerator NSXMLDTDKind was mistakenly annotated with NS_SWIFT_NAME(DTDKind) instead of NS_SWIFT_NAME(dtd). We'd like to fix that in a later version of the SDK, but we need to maintain Swift 3 compatibility on the offchance someone is using the bad name.
There's already an unversioned SwiftName API note for NSXMLDTDKind to mark it as 'dtd', so with this change we can just not mention it in the headers at all.

@swift-ci Please smoke test

so this will be a breaking change without an update to the Foundation headers right?

It's not for two different reasons:

Swift itself still defaults to Swift 3 mode, so it would only be a breaking change in Swift 4. (And one I assume is intended, given all the other enumerators.)
Until apple/swift-clang#67 gets into a Swift build, header-specified NS_SWIFT_NAMEs still win over unversioned API notes, so even in Swift 4 mode nothing changes just yet.

After this change, Foundation can either remove its NS_SWIFT_NAMEs from NSXMLDTDKind entirely, or fix it in the header and then remove the unversioned entry from the API notes. We're stuck with this new versioned one for compatibility reasons, though.

TBH I would much rather remove that ugliness from our headers (this particular case is hard to read). If we did that (removed the NS_SWIFT_NAME) it would need a synchronized submission then?
Just making sure that it isn't changing the name in swift 3 or 4 'cause that would require a full API review.

I don't think we need a fully synchronized submission, but Foundation will have to wait until this change is submitted first. Still, this is planning to change the name in Swift 4; it's just changing it to "dtd" which it was always intended to be in Swift 3. (And presumably was reviewed as such.)

@swift-ci Please smoke test Linux

@parkera should we hold off on this until proposals go through for the swift 4 naming to dtd?

If you'd prefer I can change it to "DTDKind" everywhere for now, just to get the Clang patch unblocked.

I think that would be the best route so that we can get it approved as an API change - it seems reasonable but there may be other issues yet unforeseen (iirc there is ambiguity with dtd)

Okay, does this look good?
@swift-ci Please smoke test

(ping)

Yea go for it, I will see if we can follow up on this in Foundation separately.

Thank you!


Doing snowflake snapshot via dbt archive

Issue
Try running abt archive on the snowflake, but it only archives the last day data
Issue description
Running abt archive on the snowflake, but it only archives the last day data
But what I want to do is : running abt archive every day  and keep all table  snapshot history in the db
the dbt command is sth like below :
EXECUTION_DATE=2019-01-01  dbt archive  --profiles-dir=<profiles-dir>     --profiles-dir .

the dbt_project.yml archive setting  is like this  :

archive:

    - source_schema: my_schema
      target_schema: my_archive_schema
      tables:
      - source_table: my_table
        target_table: my_archive_table
        updated_at: RECORDED_AT::TIMESTAMP
        unique_key: RECORDED_AT::TIMESTAMP || PARTNER_ID

Results
There is only last day data archived in the snowflake
e.g.
dbt archive table on 2019-01-01 --> snowflake table only has 2019-01-01 data
dbt archive table on 2019-01-02 --> snowflake table only has 2019-01-02 data
dbt archive table on 2019-01-03 --> snowflake table only has 2019-01-03 data
.
.
.

System information
The output of dbt --version:
installed version: 0.12.2
   latest version: 0.12.2

The operating system you're running on:
Mac OSX Mojave

The python version you're using (probably the output of python --version)
Python 3.5.4 :: Anaconda, Inc.

Who will this benefit?
People who want to use dbt archive doing db snapshots

hey @yennanliu - thanks for the report. The behavior you're seeing is definitely not intended.
Are you able to tell me a little more about the source table you're archiving?

Are RECORDED_AT or PARTNER_ID ever null?
Does the RECORDED_AT value change for a record when new data is loaded?

You said:

There is only last day data archived in the snowflake

Do you find that that data from previous days is deleted from the archive table?
dbt's Archival functionality should never delete data from the target table, but should instead modify the valid_from and valid_to dates for records when the source data changes. Are you able to check these valid_from and valid_to columns and determine if they are are being set appropriately?
Thanks for your help here -- these questions will help me reproduce this behavior locally. Let me know!

Hi @drewbanin
1 )  The RECORDED_AT and  PARTNER_ID are not null data in general, here is the sample :
Row  RECORDED_AT                           PARTNER_ID
1      2019-01-01 06:03:19.000            10000
2      2019-01-02 06:03:19.000           10001
3      2019-01-03 06:03:19.000           10002

No, RECORDED_AT  just means the timestamp the data was created. that record of that  column should be unchanged when new data is loaded

In my cases, valid_from is always as same as the RECORDED_AT
( as the  archive setting in dbt_project.yml) ;   and valid_to is always Null
below is part of the sql when running dbt archive I think would be the key of this operation?

insertions as (

        select
            current_data.*,
            null::TIMESTAMP as "valid_to"
        from current_data
        left outer join archived_data
          on archived_data."dbt_pk" = current_data."dbt_pk"
        where archived_data."dbt_pk" is null or (
          archived_data."dbt_pk" is not null and
          current_data."dbt_updated_at" > archived_data."dbt_updated_at" and
          archived_data."tmp_valid_to" is null
        )
    ),

    updates as (

        select
            archived_data.*,
            current_data."dbt_updated_at" as "valid_to"
        from current_data
        left outer join archived_data
          on archived_data."dbt_pk" = current_data."dbt_pk"
        where archived_data."dbt_pk" is not null
          and archived_data."dbt_updated_at" < current_data."dbt_updated_at"
          and archived_data."tmp_valid_to" is null
    )

Ok - i think part of this sounds right to me. If you're using the RECORDED_AT as the updated_at field, and RECORDED_AT is indeed immutable, then dbt will never set a valid_to date for the archived records!
Is there some other timestamp you can use (like an updated_at) that will signify that a record has changed? The alternative would be using something like current_timestamp() as the updated_at to always snapshot the entirety of the table every time archival runs. I'm not sure if that's appropriate for your use case, but I wanted to put it out there as a possibility.
Beyond these things, I'm still stumped by this part of your original issue comment:
There is only last day data archived in the snowflake, e.g.

dbt archive table on 2019-01-01 --> snowflake table only has 2019-01-01 data
dbt archive table on 2019-01-02 --> snowflake table only has 2019-01-02 data
dbt archive table on 2019-01-03 --> snowflake table only has 2019-01-03 data

The snippet you pasted in is indeed the core part of dbt's archival functionality. dbt uses this query to generate a temporary table, then it runs an update and an insert statement to "merge" the new archive snapshot into the destination table. I can't really imagine how dbt could be deleting records from the destination table.
Can you say more about how you're determining that "There is only last day data archived in the snowflake"? What kind of query are you running to determine that this is indeed the case?

Hey @drewbanin

There are only 2 timestamp columns in that table: _INSERTED_AT, and RECORDED_AT
and all records are all the same within that 2 columns

So how do I check there is only last data archived in the snowflake is :

via this  query :
select DISTINCT(RECORDED_AT::DATE) from DATAWAREHOUSE ;

then I will be able to see how many data with discrete RECORDED_AT is archived in the table

ok, I'm unsure if this is a typo or not, but you'd actually want to be querying datawarehouse.my_archive_schema.my_archive_table, right?
It would be helpful if you could share the actual values in the archive section of your dbt_project.yml file, as well as the contents of the debug queries that you're actually running!

Hi @drewbanin
Sorry for the typo. the query is datawarehouse.my_archive_schema.my_archive_table
thanks for point that out.
The setting of our dbt_project.yml is like below :
name: 'my_dbt_project'
version: '1.0'

profile: 'my_project'

source-paths: ["models"]
analysis-paths: ["analysis"]
test-paths: ["tests"]
data-paths: ["data"]
macro-paths: ["macros"]

log-path: "/tmp/log"
target-path: "/tmp/target"
clean-targets:
- "/tmp/target"
- "/tmp/dbt_modules"

quoting:
  identifier: false
  schema: false

models:
  materialized: view
  pre-hook: "{{ logging.log_model_start_event() }}"
  post-hook: "{{ logging.log_model_end_event() }}"

archive:

    - source_schema: my_schema
      target_schema: my_archive_schema
      tables:
      - source_table: my_table
        target_table: my_archive_table
        updated_at: RECORDED_AT::TIMESTAMP
        unique_key: RECORDED_AT::TIMESTAMP || PARTNER_ID

seeds:
  my_dbt:
    enabled: true
    schema: seed_data

on-run-end:
    - 'grant usage on schema "{{ target.schema }}" to PUBLIC'
    - 'grant select, references on all tables in schema "{{ target.schema }}" to PUBLIC'

The debug query I used like below  :
SELECT  RECORDED_AT::DATE, count(*) FROM  DATAWAREHOUSE.my_schema.my_table
GROUP BY 1 ORDER BY 1  ;

hey @yennanliu - gosh, I hope I'm not being pedantic here, but your debug query is still looking at the source table! I think there's some amount of user error happening here. I say this for the following reasons:

i can't think of any reason at all why archival would delete data from previous days (as you're describing)
it very much sounds to me like you're looking at the source table, and not the archived table from the info you've sent over above

The debug query you sent is:
SELECT  RECORDED_AT::DATE, count(*) FROM  DATAWAREHOUSE.my_schema.my_table
GROUP BY 1 ORDER BY 1  ;

This is querying the source data table! You want to query the archive table! This query should show you the archived records:
SELECT  RECORDED_AT::DATE, count(*)
FROM  DATAWAREHOUSE. my_archive_schema. my_archive_table
GROUP BY 1
ORDER BY 1  ;

Are you running dbt archive on a schedule to periodically snapshot this table? I'd recommend familiarizing yourself with the documentation on archival if you haven't already.
I think one of the way to check if the dbt archive works as expect is checking the valid_from and valid_to columns in the archived table?
These columns won't be super meaningful to you because the updated_at value you're using (RECORDED_AT::TIMESTAMP) is immutable for a given record. Every record in your archive table will have a valid_from date (set to the value of RECORDED_AT::TIMESTAMP) but none of these records will have a valid_to value.
dbt uses the valid_to field to invalidate an old record when when new data is received for that record.  dbt identifies new data by comparing the updated_at value for a source record to the existing updated_at value in the archived table for a given unique_key. This is all to say: with your configuration, valid_to will always be null.
Can you run this query and paste the results in a comment below?
select
  valid_from::date as valid_from_date,
  count(*) number_of_records
from datawarehouse.my_archive_schema. my_archive_table
group by 1
order by 1

@yennanliu I'm going to close this issue, but feel free to re-open if you still have questions about archival!


Deprecate and remove DoFnTester

Imported from Jira BEAM-3650. Original Jira may contain additional context.
Reported by: kenn.

The following subtask(s) are associated with this issue: #18688, #18689, #18690, #18691, #18692, #18693, #18694


unbind not invoked for custom-elements under root level repeat.for within slot content

I'm submitting a bug report

Library Version:
1.4.1

Please tell us about your environment:

Operating System:
Windows 10

Node Version:
7.50

NPM Version:
4.2.0

JSPM OR Webpack AND Version
JSPM<EMAIL_ADDRESS>

Browser:
Chrome

Language:
TypeScript 2.2

Current behavior:
This is a very obscure bug, and is best explained with an example. I'm not sure if the repeat.for is part of the root issue here or not.
  <collapsible-island>

${dmaFunction}

      <dma-mini-status if.bind="isImport" dma-id.bind="loggerLink.linkedDmaId"></dma-mini-status>
      <dma-status-logger logger-link.bind="loggerLink"></dma-status-logger>
      <dma-mini-status if.bind="isExport" dma-id.bind="loggerLink.linkedDmaId"></dma-mini-status>

  </collapsible-island>

Our custom elements <dma-mini-status> and <dma-status-logger> components have bind called on render but unbind is never called.
However if we wrap the default content in a
 (as shown below) then everything works as normal.
  <collapsible-island>

${dmaFunction}

        <dma-mini-status if.bind="isImport" dma-id.bind="loggerLink.linkedDmaId"></dma-mini-status>
        <dma-status-logger logger-link.bind="loggerLink"></dma-status-logger>
        <dma-mini-status if.bind="isExport" dma-id.bind="loggerLink.linkedDmaId"></dma-mini-status>

  </collapsible-island>

I'm a bit rushed now so a reproduction is difficult for me to produce right now but will try to follow up soon.
It appears @RWOverdijk and @t03apt are actually experiencing the same issue:  https://gitter.im/SpoonX/Dev?at=58e630e468bee3091f17068f

Does this represent your case?

@StrahilKazlachev yes, precisely. If you remove the wrapper as indicated unbind is not called. Thank you very much for putting together that gist. It is very much appreciated!

@EisenbergEffect Fixed by #592

It seems that #592 does indeed fix the issue. Unfortunately we've identified another issue in which unbind is not invoked when combined with routing. I've created a gist.run to show this.
In this scenario the subitem and slotitem class's unbind functions are not invoked when navigating, while the other component's unbind functions are called. This can lead to memory leaks since the bindings are kept alive unnecessarily.

I think the problem occurs when you have a component that has a slot whose content is a component that has a slot itself and a repeat.for is used on that component. I've created another gist.run that shows this behaviour. When you click the replace button you'll see (in the console) that unbind() isn't called on the slot-item component. Adding the dummy div around the  in the page.html as suggested above does indeed fix the problem, but it's still a bug IMHO since adding the div might cause layout issues.
In debugging this I came across this _projectionRemoveAll function and in our case the if on that line evaluates to false, thus returnToCache isn't being called which in turn causes the unbind() function not being called on the component. The question is why the isAttached is false here.

@jmezach isAttached should be false if its parent ViewSlot is already detached. I think the fix introduced didn't cover this case.

Maybe can be closed.
Pull request merged


Travis skip cleanup

Travis would build, then delete the build, and finally deploy nothing.
skip_cleanup disables workspace delete prior to deploy

Coverage remained the same at 43.951% when pulling a47193e45d5cebc38bd217e0aa3d6282c6861d21 on bjornreppen:travis-skip-cleanup into 48a3b88e520aadcb6d6f929c22a3ad63fc2e3993 on Artsdatabanken:master.


Multiple pcaps

Changes allowing multi-select in PCAP dialog and preventing errors by Mac OS resource files included in zips.

Thanks for the submission.  I'm not convinced this functionality is worthwhile.  It breaks the ability to select pcaps located in different directories.  The thinking behind the existing functionality is that if it is too onerous to use multiple dialog windows to select multiple pcaps and/or if you want to submit a large number of pcaps, just tar them (or zip them or use one of the other supported archive/compression formats).


Fix some clippy warnings

lgtm
could you also add clippy to travis?

I want to but currently blocked by https://github.com/Manishearth/rust-clippy/issues/1824


first draft at servant-0.12 support

so stackage want to migrate to servant-0.12 and we must port slack-web.
This is a first stab at it, everything seems to work, but as I wrote there, I don't know how to make slack-web build against both servant-0.11 and servant-0.12 without introducing at least a cabal flag.
In the meantime if you want you can peek at the changes. Note that I half-did it. This patch half supports servant-0.11 and half doesn't ;-) I stopped with the #ifdefs when I hit the .cabal dependency issue. Either I'll drop completely 0.11 or do something else depending on what the stackage people suggest (hopefully they'll answer).

ok so I see two options forward (see https://github.com/fpco/stackage/issues/3013#issuecomment-343141174 ) =>

we support only servant-0.12 from here on out
we support both 0.11 & 0.12 using a cabal build-time flag

i would go with the first option, less work and less to test in the future. I'll update this PR do to that, and also remove the circleCI setup to test again lts-8.4 which failed since there's no servant-0.12 there.
Let me know if you don't agree! In that case I can absolutely do the build flag solution.

@emmanueltouzery I agree with the first option (Servant 0.12). 👍

Great! I asked something more about the stack.yml on the stackage thread, but i think what's in the branch now is ok. I'll probably merge tomorrow and upload a new point release on hackage. I may have to release another one when stackage moves to servant-0.12 due to the stack.yml but if i do it'll be trivial.


[Improvement] Gravitino Web UI shows PostgreSQL password

What would you like to be improved?
Showing the password could be a security issue.

How should we improve?
Perhaps don't show the password?

Is the return of this field something the backend should remove or the frontend should hide? I think it's better for the backend to remove the field when returning data.

It should be done by the backend, not the UI.
@Clearvive can you please work on this.

@justinmclean @jerryshao is this done? can i pick it up?

@ajayl83 I think someone is already working on this. You can pick other issues.

@ajayl83 I know it's been a while but do you want to work on this?

Hi @justinmclean,
I found this issue at the bottom of the "good first issue" list, and it seems like it has been pending for a while.
I'd like to try solving this. I noticed that the files CatalogResponse and CatalogListResponse directly return CatalogDto, which includes sensitive information.
My proposed solution is to add a withMaskedCredentials function to CatalogDto. Then, when creating CatalogResponse and CatalogListResponse, they would use CatalogDto.withMaskedCredentials instead of directly returning CatalogDto.
Could you let me know if this approach is okay? If not, would you please guide me toward a better solution?

public CatalogDTO withMaskedCredentials() {
    if (properties == null || properties.isEmpty()) {
      return this;
    }

    Map<String, String> maskedProperties =
        properties.entrySet().stream()
            .collect(
                Collectors.toMap(
                    Map.Entry::getKey,
                    e -> e.getKey().toLowerCase().contains("password") ? "***" : e.getValue()));

    return new CatalogDTO(name, type, provider, comment, maskedProperties, audit);
  }


Add scrollViewWillEndDragging delegate

Changes in this pull request
There was no way to implement this useful scroll view delegate when using an IGListAdapter since it wasn't being passed through. This change simply adds in the missing delegate.
Don't really need any tests since this is just a passthrough delegate, similar to how scrollViewDidScroll is being passed through.
Checklist

[x] All tests pass. Demo project builds and runs.
[x] I added tests, an experiment, or detailed why my change isn't tested.
[x] I added an entry to the CHANGELOG.md for any breaking changes, enhancements, or bug fixes.
[x] I have reviewed the contributing guide

@xezero is this fixing a bug? We use this delegate API in Instagram and it seems to be working fine 🤔
We use an NSProxy subclass to split & forward methods between UIScrollViewDelegate and UICollectionViewDelegate. If IGListAdapter implements them, we send the method to the adapter, otherwise we forward to the delegate.
If you can accompany this with a failing unit test that passes w/ these changes, happy to merge! Not sure this is an issue though.

The NSProxy subclass forwards the methods through but IGListAdapter doesn’t pass scrollViewWillEndDragging through to the scrollViewDelegate. It only seems like its forwarding 4 of them (scrollViewDidScroll, scrollViewDidEndDragging, etc).

@xezero the proxy only sends methods to the adapter when it implements them, otherwise methods are sent to the scroll delegate.
If you can provide a test or example app showing an bug I’m happy to take a look!
Sent with GitHawk

Closing as I'm not sure how this fixes anything and we need a failing test first.

Hey @rnystrom  @xezero  can you help me understand this
"We use an NSProxy subclass to split & forward methods between UIScrollViewDelegate and UICollectionViewDelegate. If IGListAdapter implements them, we send the method to the adapter, otherwise we forward to the delegate."
Thank you in advance.
Regards.


Prevent dangling cat-files                                           …

In #17138 it appears that on Windows a git cat-file can fail to be killed.
This PR attempts to prevent this by:

Using os.Pipes for the input - this should mean that the stdin is definitely closed.
If the cat-file is doesn't close properly it will attempt to kill it repeatedly logging this.

Fix #17138
Signed-off-by: Andrew Thornton<EMAIL_ADDRESS>

This is WIP because the current mechanism for killing processes is racy!

data race detected

This doesn't work and the original issue appears to have solved itself.


Add missing Identity field to VerifyEmail job options

🔧 Changes

This PR adds an optional Identity field to the management.Job struct, for use in the (*JobManager).VerifyEmail method.
Per the docs, this field "must be provided to verify primary social, enterprise and passwordless email identities. Also, is needed to verify secondary identities.".
This field only accepts a subset of the UserIdentity fields, so I have implemented (*Job).MarshalJSON to allow the reuse of the UserIdentity struct whilst omitting the extraneous fields.
🔬 Testing

I have added the Identity parameter to TestJobManager_VerifyEmail and re-recorded the HTTP interactions, so this is covered by the end-to-end tests.
📝 Checklist

[x] All new/changed/fixed functionality is covered by tests
[x] I have added documentation for all new/changed functionality

Codecov Report
Patch coverage: 100.00% and project coverage change: +0.01 :tada:

Comparison is base (41597ea) 94.90% compared to head (f90b387) 94.92%.

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #197      +/-   ##
==========================================
+ Coverage   94.90%   94.92%   +0.01%
==========================================
  Files          38       38
  Lines        6891     6915      +24
==========================================
+ Hits         6540     6564      +24
  Misses        280      280
  Partials       71       71

Impacted Files
Coverage Δ

management/job.go
57.95% <100.00%> (+11.57%)
:arrow_up:

management/management.gen.go
100.00% <100.00%> (ø)

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.

Nice one, thanks @ewanharris!


Install task fails if test:prepare task does not exist

If a Rails app does not have test:prepare defined, the install task fails with:
rails aborted!
Don't know how to build task 'test:prepare' (See the list of available tasks with `rails --tasks`)
Did you mean?  db:test:prepare

The most common path to get into this state is likely generating a new Rails app with -T to skip the default test framework (perhaps in favor of rspec or another alternative)
rails new no_tests -T
cd no_tests
bundle add jsbundling-rails
rails javascript:install:esbuild #errors out

The same issue exists on cssbundling-rails
A simple fix is something like:
# lib/tasks/jsbundling/build.rake
Rake::Task["test:prepare"].enhance(["javascript:build"]) if Rake::Task.task_defined?("test:prepare")

Happy to open a PR to resolve this error and on cssbundling-rails, if this is something that you would like to address.

Please do 👍


Output Folder Generated Without Names

Hello,
I am currently experiencing an issue with the generation of output folders in the program. The folder creation appears to be based on the search term that is used. However, when the search term is in a language other than English (e.g., Korean), the output folder is created without a name.
This appears to occur due to the following line of code from research_agent.py:
self.directory_name = ''.join(c for c in question if c.isascii() and c not in string.punctuation)[:100]

This code seems to delete the folder name if it includes characters that are non-ASCII or punctuations. Consequently, when the search term is in Korean, the output folder is created without a name because all the characters are deleted by this line of code.
I believe this is a bug, as folder names should be created regardless of the language of the search term.
Could you review this issue and consider a fix or an update to address it?
Thank you in advance for your attention to this matter.
Best regards,

Hi @fbwndrud ! Thank you for pointing it out. The idea behind that line is to keep the file paths valid so the user could safely reach the pdf at the end of the research. I will dive deeper into finding a solution that could work for all cases, maybe a random tmp folder name.
for now, you can locally change this line and maybe use str(uuid.uuid4()) for the directory name.
Thank you!


[RFC] Provide an accessible focus alternative when disableRipple property on the MuiButtonBase is on!

What's the problem? 🤔
When you use the disableRipple: true, property on the theme.components as such
    MuiButtonBase: {
      defaultProps: {
        disableRipple: true,
      },
    },

Then all of the components that rely on the Mui-focusVisible class for ripple styling become unaccessible. When you focus them with keyboard nothing happens, you can't tell they are focused.

Ripple
Disabled Ripple

What are the requirements? ❓
There needs to be some kind of fallback on that. I did that in my app and was left scratching my head for a while, why nothing is working, including:

Radio Buttons
Checkboxes
Buttons

What are our options? 💡
What I have done in my app is a global style to the:
.Mui-focusVisible: {
outline: 2px solid #000
}

There should be some fallback
Proposed solution 🟢
I am unaware of the internals of MUI and someone with more knowledge should propose a solution. My solution to my application is what I wrote in the "What are our options? 💡" section.
Resources and benchmarks 🔗
No response

@dimitur2204 Sorry for the dull question but why do we want to disable the standard ripple effect instead of customizing it to our needs for example?

@dimitur2204 Sorry for the dull question but why do we want to disable the standard ripple effect instead of customizing it to our needs for example?

That I don't know. It was disabled all the way back when I started work on the repository from what I remember 😁

@dimitur2204 Sorry for the dull question but why do we want to disable the standard ripple effect instead of customizing it to our needs for example?
Here's a simple example https://codesandbox.io/s/mui-change-global-ripple-color-forked-zk0xr3

@kachar The ripple controls both the click ripple and the focus ripple. It's a common case to not want the click ripple, but by doing that you render everything that uses the focus ripple inaccessible

@michaldudak It seems like it is stated in the API for the Button Base https://mui.com/material-ui/api/button-base/#props.

As you noticed, it is an expected and documented behavior. If you disable the ripple, you have to provide your own styles for the focus state.
We can't really do much to this in v5, as it may break users who depend on this, but we are going to change this in v6 (by replacing the focus ripple with a different background shade).


Inconsistency in hotel room prices

Steps to reproduce:

search hotels for 5 adults, you'll see hotel for let's say 1940 EUR
open this hotel

Now there are two situations:

you are able to find a room with this price for 5 adults, in this case, everything is fine and you need to find another hotel:
you'll be able to find cheaper rooms (usually for less than 5 adults)

Questions to answer:

[ ] Why is there this inconsistency? Is it normal behavior of Booking.com? I can imagine an explanation where this is correct.
[ ] Can we improve it somehow to prevent this confusion?
[ ] Are we actually sending information about adults to the single hotel page? This needs verification.

Example 1: I found a hotel room for 5 adults and price 1010,32 but after opening the hotel - there is no such a room. However, there are two smaller rooms for 442,67 and 567,65 (1010,32 in total) so from my point of view, it works correctly.
Example 2: City "Zlín", hotel "Penzion Sokolská", two persons - price is 300 but there are even cheaper rooms for 2 adults. Why?
See: https://trello.com/c/fB6jWpxI/112-bug-inconsistency-in-prices-on-search-and-in-hotel-booking

I checked the exact same dates from Example 2 on Booking.com. The price in the search result said 300 EUR, but the cheapest room was actually 228 EUR.
I don't think we can do anything here? Can we register a bug with Booking.com?


Comdat support

Is threre any chance of support for COMDAT support in Object?
ELF supports "any" comdat (linker may pick any match)
COFF supports several, (full list https://llvm.org/docs/LangRef.html#comdats )
Essentially this is useful to "bind" things together, if 1 is linked, the other thing is linked too, despite not having a symbol reference or relocation to it.

I assume this is for write support to be used in cranelift.
It's definitely something that we should support. I'm not familiar with it though, and can't yet give much advice on what it would involve, but it sounds like we need the ability to define a list of comdat keys in the object, and the ability to set a comdat key for each symbol (or alternatively, define a list of symbols for each comdat key).

Maybe llvms's approach is most flexible:
There's a list of COMDAT's, which has a name and comdat mode (any, exactmatch, largest, noduplicates, samesize)
And a variable / function (Read: section) can optionally refer to it.
ELF then has the restriction that only "any" is supported, and COFF requires the comdat key to match the name of one of the symbols that use it.

So I did some more research:

https://docs.rs/object/0.20.0/object/enum.SymbolFlags.html << You already have COFF comdat covered.

I'll update this post when I figure out how ELF comdat works.

I think we want to provide a unified API for comdat, rather than using the specific file format settings in SymbolFlags etc.
I got as far as generating sample COFF and ELF files with comdat sections, but didn't start designing the API yet.

It looks like MSVC and GNU differ in how they implement COMDAT. Specifically, GNU doesn't appear to use IMAGE_COMDAT_SELECT_ASSOCIATIVE, and I don't understand yet what they are doing instead. LLVM appears to follow the MSVC method. Do you need to support both of these methods?

I personally need the 'llvm' / msvc method


feat: Menu actions

Feature Request
Feature Proposal
Link buttons to page sections.
The expected behavior is that clicking will scroll to the selected section.

@Guboehm

@Guboehm hello, I already inserted the links in the items that were already ready, I also talked to Heron and requested the branch that contains the missing items to complete the Landing page and thus finish all the links.

Nice, we are looking forward to the pull request.