Failed to send attachment with mail

I can send mail through SMTPS. But, unable to attach any file with the mail. Although the code below compiles fine, but I am getting mail without attachments. The Java version of the following flow is working alright. I can't figure out what's wrong I did in the Karavan conversion.
This is my flow:
apiVersion: camel.apache.org/v1
kind: Integration
metadata:
  name: mail-with-attachment
spec:
  flows:
    - route:
        from:
          uri: timer:test
          steps:
            - pollEnrich:
                expression:
                  simple:
                    expression: file:file_src?noop=true
            - log:
                message: 'polled: ${body}'
            - setBody:
                expression:
                  exchangeProperty:
                    expression: |-
                      exchange -> {
                          byte[] fileBytes = exchange.getIn().getBody(byte[].class);
                          org.apache.camel.attachment.AttachmentMessage attMsg = exchange.getIn(org.apache.camel.attachment.AttachmentMessage.class);
                          attMsg.addAttachment("report.pdf", new DataHandler(fileBytes, "application/pdf"));
                          return exchange.getMessage().getBody();
                      }
            - setHeader:
                name: subject
                expression:
                  simple:
                    expression: Test subject from Karavan
            - setHeader:
                name: to
                expression:
                  simple:
                    expression: '{{mail.to}}'
            - setBody:
                expression:
                  constant:
                    expression: >-
                      Hello,<br>This is test body from Karavan with an
                      attachment.
            - log:
                message: 'PDF_SENDER_ROUTE: mail body: ${body}'
            - toD:
                uri: >-
                  smtps://{{mail.host}}:{{mail.port}}?password={{mail.password}}&username={{mail.username}}&contentType={{mail.content-type}}
            - log:
                message: 'PDF_SENDER_ROUTE: Mail Sent'
                loggingLevel: ERROR
          parameters:
            delay: '1000'
            period: '10000'

This is my properties file:
camel.jbang.project-id=mail_with_attachment
camel.jbang.project-name=mail_with_attachment
camel.jbang.project-description=mail_with_attachment
camel.jbang.gav=org.demo:mail-with-attachment:1.0
camel.jbang.runtime=quarkus
camel.jbang.dependencies=mvn:io.quarkus:quarkus-container-image-jib,camel:microprofile-health,mvn:io.quarkus:quarkus-openshift,camel:file,camel:mail
camel.health.enabled=true
camel.health.exposure-level=full
camel.jbang.quarkusVersion=2.11.2.Final
quarkus.kubernetes-client.trust-certs=true
quarkus.container-image.builder=jib
quarkus.container-image.group=karavan
quarkus.container-image.name=mail_with_attachment
quarkus.openshift.route.expose=false
quarkus.openshift.part-of=mail_with_attachment
quarkus.openshift.replicas=1

mail.content-type=text/html
<EMAIL_ADDRESS>mail.password=<password>
<EMAIL_ADDRESS>mail.host=smtp.gmail.com
mail.port=465
<EMAIL_ADDRESS>
Note: All files in file_src folder are PDF
Java Version: 11
Camel Version: 3.18.2
Karavan Version: 3.18.2
Run command: jbang -Dcamel.jbang.version=3.18.2 camel@apache/camel run *
I need some assistance. Please help.

You need to use joor language when inlining java code (it uses joor-java)
                expression:
                  joor:
                    expression: >-
                      byte[] fileBytes = exchange.getIn().getBody(byte[].class);

                      org.apache.camel.attachment.AttachmentMessage attMsg =
                      exchange.getIn(org.apache.camel.attachment.AttachmentMessage.class);
                          attMsg.addAttachment("report.pdf", new javax.activation.DataHandler(fileBytes, "application/pdf"));
                      return exchange.getMessage().getBody();

PS: We may rename joor to just java to make it less confusing.
Also you need to use fully qualified class names which you almost had, try with above ^^^

@mgubaidullin btw the "expression" panel in karavan needs that [ ] to make it big when you edit source code like above ^^^^

Like this?

I would put this kind of code to a bean instead of expression

Ah yeah you can already do it, but its not as obvious as the others have that [ ] icon with "change to text area".

yeah but people would often in a prototype put all the code in the same file.
If you put it in a .java file you need all that public class ... stuff

you can use groovy as language and then have code similar to what the user did - eg its just script inlined in a function.
But if you want to use regular java code compiled with java compiler then you need all the boilerplate with class / method and to call it via bean expression ala #class:com.foo.MyBean -- that is way overhead for 3 lines of code.

Something like that?

Yeah that is great - and could you maybe also show what language you have selected, in the editor
Expression (joor language)

Thanks for the assistance. @davsclaus @mgubaidullin


UnsafeCallOnNullableType not reporting !! usage

Expected Behavior
UnsafeCallOnNullableType rule should report usages of !! as an unsafe call on nullable type.
Observed Behavior
UnsafeCallOnNullableType rule isn't reporting !! as an unsafe call on nullable type after 1.1.0 update.
Steps to Reproduce
When running the following test:
class UnsafeCallOnNullableTypeTest {
    private val code: String =
            """
                val aString: String?
                aString = null
                val aStringLength = aString!!.length
            """.trimIndent()

    @Test
    fun shouldReportDoubleBangUsage() {
        val findings = UnsafeCallOnNullableType().lint(code)

        assertThat(findings).hasSize(1)
        assertThat(findings[0].message).isEqualTo("Calling !! on a nullable type will throw a " +
                "NullPointerException at runtime in case the value is null. It should be avoided.")
    }
}

The test won't pass if the UnsafeCallOnNullableType rule after f80fe60383c78bed811cdad1fd54402e21b58a11 commit is used but, if you remove the 3 lines added to the rule by the linked commit or use version 1.0.1, it will pass.
Your Environment

Version of detekt used: 1.1.1
Version of Gradle used (if applicable): 5.6.2
Operating System and version: Ubuntu Mate 18.04
Link to your project (if it's a public repository): Private Repository

The behaviour of the rule has changed to now require type resolution, which means if you're running detekt without type resolution (not passing a classpath) then it won't run the rule.
We need docs for this and have an issue open already to cover that.
If you're actually running detekt with type resolution and the rule is still not picking up this case then we'll need to look more closely at the problem.

By passing a classpath you mean running detekt with --classpath flag? If yes, could you link an example?

One way would be running gradlew detektMain with debug output and looking for the flags passed to detekt when it runs.

I ran the task gradlew detekt since I didn't have the task detektMain with debug output and couldn't find any classpath flag being passed to detekt.
Could you please clarify how should I do this? Is there a way to declare it inside my detekt block like toolVersion, input, filters, etc?

Gradle creates tasks for each source set but currently only on Kotlin JVM projects, is it maybe an Android, native or JS project?
We definitely need docs for this so probably creating those is the action we need to take to resolve this, see #1943

It's an Android project.

Yeah we're not generating required Gradle tasks for Android yet. It's on my to do list; I'll work on it next since there's a clear need.

The required Gradle tasks for Android are described in another issue. Therefore, I’ll close this.

@schalkms Can you link the issue with this TODO?

Link #2259


Feature Request: Adding the AGENT_SECRET environment variable inside of the config

Problem/Motivation
Currently there isn't a good way to add the AGENT_SECRET inside of the portainer config. This Env Var allows you to be able to connect to remote portainer agents on external docker servers, only if the portainer agent also has the same shared secret.
This secret only takes affect when one the agent is restarted, because the agent utilized a trust first connection approach. This is bad if a malicious actor manages to connect to the agent first. To combat this they added the shared AGENT_SECRET env var, which will cause the agent to only pair with the first connection that has the shared secret as well.
As always thank you for all of your great work!

Merged in dev, will be part of the next release


Simplify lock and remote update

The complexity of lock could be avoided if a command e.g. a command shelephant remote_update myname would exist. In reality only .shelephant/storage/myname is needed.
One could even consider having a .shelephant_remote folder instead to make a distinct difference

Another question is if lock should have the possibility to allow multiple names, if the data is part of multiple datasets


Issue with the New EnderIo Update

Issue Description:
As soon as I launched it crashes at 4 out of 7
Delete this line after reading it: If this bug occurs in a modpack, please report this to the modpack author first. If you are using an unreleased version of Ender IO (a.k.a development/nightly version), please don't open an issue for features that have not been announced as "finished" or "working". Expect to be ignored if you ignore this template.
What happens:
Launch game and it crashes
What you expected to happen:
a fix
Steps to reproduce:

Updated Ender Io in my pack
2.wait till like 4 out of 7
then crashes
...

Affected Versions (Do not use "latest"):

EnderIO: 1.12.2-5.1.54
EnderCore:1.12.2- 0.5.73
Minecraft: 1.12.2
Forge: <IP_ADDRESS>47
SpongeForge? yes/no no
Optifine? yes/no no
Single Player and/or Server?

Your most recent log file where the issue was present:
https://paste.dimdev.org/cedorixene.mccrash
[pastebin/gist/etc link here]

-> EIOP


Adding script to remove rancher-webhook

Rancher webhook is now in downstream clusters as of 2.7.2 This can cause issues on rollback to earlier versions, so a script was created to help delete these webhooks

Related to #40816, will need to get this in before 2.7.2

@aemneina Any chance you would be willing to give this a review?


persistence bbolt plugin build problem

I try config volantmq with plugin, but build plugin is success, but unable to run.
Error: plugin was built with a different version of package gopkg.in/yaml.v3
I am not sure where is the problem.

volantmq go.mod is required to updated .
gopkg.in/yaml.v3 v3.0.0-20200121175148-a6ecf24a6d71

=》
gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c
The problem solved.

thanks, i'll update


Fix functional types

Correct me if I'm wrong, but if a function doesn't explicitly return nil, it returns nothing, which are different things? I see a lot of functional types written as () -> nil but I believe it's supposed to be written as () -> (). I'm not 100% sure though.

You are correct that there is a difference between nil and "nothing." I'm not sure what distinction Luau makes here either though

() -> () is the correct way to represent a function that returns "nothing," so we can move forward with swapping out functions that are currently annotated as returning nil when they actually don't


using Keyborad.remove()

Actually when the go button(in land scape Orientation) is pressed then there is
a try and catch statement in RealmScreen.js which is asynchronous so we have
to wait but ideal behavior must be this that whatever be the case first of all
keypad must be closed in a synchronized  manner so Keyborad.remove() must be
called before the try catch statement.
https://user-images.githubusercontent.com/56453541/104292591-0568dd80-54e3-11eb-8eee-25e0ce212228.mp4
Fixes: #4386

@chrisbobbe kindly plz review this PR

@chrisbobbe kindly plz review this PR

(Discussion in chat.)

(Discussion in chat.)

Superseded by https://github.com/zulip/zulip-mobile/commit/eb204c43d0df391fe82656fdbda5aa9064ed4d11 .


[incubator/ElasticSearch] is unstable on cluster provisioned by acs-engine

Is this a request for help?:

Is this a BUG REPORT or FEATURE REQUEST? (choose one): BUG REPORT

Version of Helm and Kubernetes: Helm 2.5.1 Kubernetes 1.7.4, running in Azure, provisioned by ACS-Engine
Which chart: incubator/ElasticSearch
What happened: When I interact with ElasticSearch (or Kibana) at any sort of pace above one-offs, the elasticsearch-data-[0-2] pods get into a bad state. Once I stop the activity, the pods eventually recover.
What you expected to happen: I would expect ES to be able to keep up with moderate loads; it can't even keep up with light load.
How to reproduce it (as minimally and precisely as possible): Install the chart, then install the fluentd for Kubernetes daemonset: https://github.com/fluent/fluentd-kubernetes-daemonset
Anything else we need to know: Not sure the appropriate place to file this issue; may cross-link it to the acs-engine github as well.

The most recent version seems to be much more stable. Closing this.


[Operator] Add index_add or index_update to numpy extension

We need the functionality to calculate b = index_add(a, indices, value), which mimics the outcome of a[indices] += value.
This is similar to the tensor_scatter_nd_add in TF: https://www.tensorflow.org/api_docs/python/tf/tensor_scatter_nd_add
Also in JAX: https://jax.readthedocs.io/en/latest/_autosummary/jax.ops.index_update.html#jax.ops.index_update

Besides that, it would be great if npx.scatter_nd is implemented for mx.np.ndarray.

@ZheyuYe @JiangZhaoh @yzhliu @haojin2
To understand the problem, let's consider two use cases. The first one can be solved via gather_nd and the second one cannot be solved via the existing MXNet.
Take elements at specific locations from the input data
out[i, j, ...] = data[i, positions[i, j], ...]
In GluonNLP, the positions are masked locations in the input that we will need to calculate the loss. data is the mapped hidden states of the sequences.
With advanced indexing + imperative API, we can do something like this:
import mxnet as mx
mx.npx.set_np()

data = mx.np.random.normal(0, 1, (5, 5, 5, 5))
positions = mx.np.random.randint(0, 5, (5, 4))
out = data[mx.np.expand_dims(mx.npx.arange_like(data, axis=0), axis=-1), positions]
print(out.asnumpy().shape)

In order to make the network hybridizable, we can implement it via gather_nd:
@use_np
def select_vectors_by_position(F, data, positions):
    """Select each batch with the given positions.
    Once advanced indexing can be hybridized, we can revise the implementation.
    out[i, j, :] = data[i, positions[i, j], :]
    Parameters
    ----------
    F
    data
        Input tensor of contextualized token embeddings
        Shape (batch_size, seq_length, units)
    positions
        Input tensor of the positions.
        Shape (batch_size, num_sel_positions).
        For each sample in the batch, the values in this tensor must not exceed
        the length of the sequence.
    Returns
    -------
    out
        The selection result.
        Shape (batch_size, num_sel_positions, units)
    """
    # Here, we use gather_nd to select the output from data:
    # Need to compute
    #   out[i, j, :] = in[i, masked_position[i, j], :]
    # Thus, construct a indices with shape [2, batch_size, num_masked_position], where
    #     indices[0, i, j] = i
    #     indices[1, i, j] = masked_position[i, j]
    # Then, out = gather_nd(in, indices)
    positions = positions.astype(np.int32)
    # batch_idx.shape = (batch_size, 1) as [[0], [1], [2], ...]
    batch_idx = F.np.expand_dims(F.npx.arange_like(positions, axis=0),
                                 axis=1).astype(np.int32)
    batch_idx = batch_idx + F.np.zeros_like(positions)
    indices = F.np.stack([batch_idx, positions])
    out = F.npx.gather_nd(data, indices)
    return out

Update elements at specific locations of the input data
For example, if we need some selected locations and will need to replace the elements without own generated element, i.e.,
data[i, positions[i, j], ...] = update_val[i, j, ...]
With advanced indexing + imperative API, we can do something like this:
import mxnet as mx
import numpy.testing as npt
mx.npx.set_np()

data = mx.np.random.normal(0, 1, (5, 5, 5, 5))
positions = mx.np.random.randint(0, 5, (5, 4))
update_val = mx.np.random.normal(0, 1, (5, 4, 5, 5))
data[mx.np.expand_dims(mx.npx.arange_like(data, axis=0), axis=-1), positions] = update_val
print(out.asnumpy().shape)

# or do
data[mx.np.expand_dims(mx.npx.arange_like(data, axis=0), axis=-1), positions] += update_val
print(out.asnumpy().shape)

However, we cannot surround it with autograd

import mxnet as mx
import numpy.testing as npt
mx.npx.set_np()

data = mx.np.random.normal(0, 1, (5, 5, 5, 5))
positions = mx.np.random.randint(0, 5, (5, 4))
update_val = mx.np.random.normal(0, 1, (5, 4, 5, 5))
data.attach_grad()
with mx.autograd.record():
   data[mx.np.expand_dims(mx.npx.arange_like(data, axis=0), axis=-1), positions] = update_val
mx.npx.waitall()

Error message:
MXNetError: Traceback (most recent call last):
  File "src/imperative/imperative.cc", line 203
MXNetError: Check failed: AGInfo: :IsNone(*output): Assigning to NDArrays that are already in a computational graph will cause undefined behavior when evaluating gradients. Please call backward first to clear the graph or do this out side of a record section. Also note that you cannot use inplace operations like +=, *=, relu(x, out=x), y[idx]=x, etc inside a record section.

We will need to have a workaround solution to this use case.

The first use case would be improved though #18319 that insipred by #17327.


Problems installing packages

I cannot install the packages. Any channel I should add?
Windows 10.
Thanks
conda install --file C:\Users\e337652\Documents\GitHub\batch2-BLU04\requirements.txt

Solving environment: failed

PackagesNotFoundError: The following packages are not available from current channels:

  - appnope
  - jupyter-core
  - prompt-toolkit
  - ptyprocess
  - ipython-genutils
  - line-profiler
  - jupyter-client
  - jupyter-console

Current channels:

  - https://conda.anaconda.org/conda-forge/win-64
  - https://conda.anaconda.org/conda-forge/noarch
  - https://repo.continuum.io/pkgs/main/win-64
  - https://repo.continuum.io/pkgs/main/noarch
  - https://repo.continuum.io/pkgs/free/win-64
  - https://repo.continuum.io/pkgs/free/noarch
  - https://repo.continuum.io/pkgs/r/win-64
  - https://repo.continuum.io/pkgs/r/noarch
  - https://repo.continuum.io/pkgs/pro/win-64
  - https://repo.continuum.io/pkgs/pro/noarch
  - https://repo.continuum.io/pkgs/msys2/win-64
  - https://repo.continuum.io/pkgs/msys2/noarch

@pma2  My approach was just to install the ones needed for this BLU 04 (provided the current environment already runs previous blus), which I think are:
xlrd
line_profiler
lxml

with conda install, and it went fine.

Ok, I did that and worked for those 3 packages ;) Thanks


MAC Address not available via SDK

In December, Intune added Ethernet MAC Address as a property (finally!). It's available via hardware properties in the portal, but if I retrieve devices using:
$a = Get-IntuneManagedDevice | Get-MSGraphAllPages
I get an empty ethernetMacAddress field on all devices. I am able to get the data if i directly query via REST like so:
$DeviceID = <some Device's ID>
$URI = [uri]::EscapeUriString("https://graph.microsoft.com/beta/deviceManagement/manageddevices('$deviceid')?`$select=UserPrincipalName,emailAddress,deviceName,model,manufacturer,serialNumber,id,lastSyncDateTime,ethernetMacAddress&_=1576515130721")
$dev = Invoke-MSGraphRequest -url $uri -HttpMethod get

Then i am able to get the ethernetMacAddress property
If I do:
Get-IntuneManagedDevice -Select id,ethernetMacAddress | Get-MSGraphAllPages
I get:
Get-DeviceManagement_ManagedDevices : Cannot validate argument on parameter 'Select'. The argument "ethernetMacAddress"
does not belong to the set "id,userId,deviceName,managedDeviceOwnerType,deviceActionResults,enrolledDateTime,lastSyncDateTim
e,operatingSystem,complianceState,jailBroken,managementAgent,osVersion,easActivated,easDeviceId,easActivationDateTime,azureA
DRegistered,deviceEnrollmentType,activationLockBypassCode,emailAddress,azureADDeviceId,deviceRegistrationState,deviceCategor
yDisplayName,isSupervised,exchangeLastSuccessfulSyncDateTime,exchangeAccessState,exchangeAccessStateReason,remoteAssistanceS
essionUrl,remoteAssistanceSessionErrorDetails,isEncrypted,userPrincipalName,model,manufacturer,imei,complianceGracePeriodExp
irationDateTime,serialNumber,phoneNumber,androidSecurityPatchLevel,userDisplayName,configurationManagerClientEnabledFeatures
,wiFiMacAddress,deviceHealthAttestationState,subscriberCarrier,meid,totalStorageSpaceInBytes,freeStorageSpaceInBytes,managed
DeviceName,partnerReportedThreatState,deviceConfigurationStates,deviceCategory,deviceCompliancePolicyStates" specified by
the ValidateSet attribute. Supply an argument that is in the set and then try the command again.
At line:1 char:38
+ $a = Get-IntuneManagedDevice -Select id,ethernetMacAddress | Get-MSGr ...
+                                      ~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : InvalidData: (:) [Get-DeviceManagement_ManagedDevices], ParameterBindingValidationException
    + FullyQualifiedErrorId : ParameterArgumentValidationError,Microsoft.Intune.PowerShellGraphSDK.PowerShellCmdlets.Get_De
   viceManagement_ManagedDevices

In December, Intune added Ethernet MAC Address as a property (finally!). It's available via hardware properties in the portal, but if I retrieve devices using:
$a = Get-IntuneManagedDevice | Get-MSGraphAllPages
I get an empty ethernetMacAddress field on all devices. I am able to get the data if i directly query via REST like so:
$DeviceID = <some Device's ID>
$URI = [uri]::EscapeUriString("https://graph.microsoft.com/beta/deviceManagement/manageddevices('$deviceid')?`$select=UserPrincipalName,emailAddress,deviceName,model,manufacturer,serialNumber,id,lastSyncDateTime,ethernetMacAddress&_=1576515130721")
$dev = Invoke-MSGraphRequest -url $uri -HttpMethod get

Then i am able to get the ethernetMacAddress property
If I do:
Get-IntuneManagedDevice -Select id,ethernetMacAddress | Get-MSGraphAllPages
I get:
Get-DeviceManagement_ManagedDevices : Cannot validate argument on parameter 'Select'. The argument "ethernetMacAddress"
does not belong to the set "id,userId,deviceName,managedDeviceOwnerType,deviceActionResults,enrolledDateTime,lastSyncDateTim
e,operatingSystem,complianceState,jailBroken,managementAgent,osVersion,easActivated,easDeviceId,easActivationDateTime,azureA
DRegistered,deviceEnrollmentType,activationLockBypassCode,emailAddress,azureADDeviceId,deviceRegistrationState,deviceCategor
yDisplayName,isSupervised,exchangeLastSuccessfulSyncDateTime,exchangeAccessState,exchangeAccessStateReason,remoteAssistanceS
essionUrl,remoteAssistanceSessionErrorDetails,isEncrypted,userPrincipalName,model,manufacturer,imei,complianceGracePeriodExp
irationDateTime,serialNumber,phoneNumber,androidSecurityPatchLevel,userDisplayName,configurationManagerClientEnabledFeatures
,wiFiMacAddress,deviceHealthAttestationState,subscriberCarrier,meid,totalStorageSpaceInBytes,freeStorageSpaceInBytes,managed
DeviceName,partnerReportedThreatState,deviceConfigurationStates,deviceCategory,deviceCompliancePolicyStates" specified by
the ValidateSet attribute. Supply an argument that is in the set and then try the command again.
At line:1 char:38
+ $a = Get-IntuneManagedDevice -Select id,ethernetMacAddress | Get-MSGr ...
+                                      ~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : InvalidData: (:) [Get-DeviceManagement_ManagedDevices], ParameterBindingValidationException
    + FullyQualifiedErrorId : ParameterArgumentValidationError,Microsoft.Intune.PowerShellGraphSDK.PowerShellCmdlets.Get_De
   viceManagement_ManagedDevices

I tried your example of
$DeviceID = <some Device's ID>
$URI = [uri]::EscapeUriString("https://graph.microsoft.com/beta/deviceManagement/manageddevices('$deviceid')?`$select=UserPrincipalName,emailAddress,deviceName,model,manufacturer,serialNumber,id,lastSyncDateTime,ethernetMacAddress&_=1576515130721")
$dev = Invoke-MSGraphRequest -url $uri -HttpMethod get

I get the error message: Parsing OData Select and Expand failed: Could not find a property named 'ethernetMacAddress' on type 'microsoft.graph.managedDevice'.

Are you sure this property exists?

In December, Intune added Ethernet MAC Address as a property (finally!). It's available via hardware properties in the portal, but if I retrieve devices using:
$a = Get-IntuneManagedDevice | Get-MSGraphAllPages
I get an empty ethernetMacAddress field on all devices. I am able to get the data if i directly query via REST like so:
$DeviceID = <some Device's ID>
$URI = [uri]::EscapeUriString("https://graph.microsoft.com/beta/deviceManagement/manageddevices('$deviceid')?`$select=UserPrincipalName,emailAddress,deviceName,model,manufacturer,serialNumber,id,lastSyncDateTime,ethernetMacAddress&_=1576515130721")
$dev = Invoke-MSGraphRequest -url $uri -HttpMethod get

Then i am able to get the ethernetMacAddress property
If I do:
Get-IntuneManagedDevice -Select id,ethernetMacAddress | Get-MSGraphAllPages
I get:
Get-DeviceManagement_ManagedDevices : Cannot validate argument on parameter 'Select'. The argument "ethernetMacAddress"
does not belong to the set "id,userId,deviceName,managedDeviceOwnerType,deviceActionResults,enrolledDateTime,lastSyncDateTim
e,operatingSystem,complianceState,jailBroken,managementAgent,osVersion,easActivated,easDeviceId,easActivationDateTime,azureA
DRegistered,deviceEnrollmentType,activationLockBypassCode,emailAddress,azureADDeviceId,deviceRegistrationState,deviceCategor
yDisplayName,isSupervised,exchangeLastSuccessfulSyncDateTime,exchangeAccessState,exchangeAccessStateReason,remoteAssistanceS
essionUrl,remoteAssistanceSessionErrorDetails,isEncrypted,userPrincipalName,model,manufacturer,imei,complianceGracePeriodExp
irationDateTime,serialNumber,phoneNumber,androidSecurityPatchLevel,userDisplayName,configurationManagerClientEnabledFeatures
,wiFiMacAddress,deviceHealthAttestationState,subscriberCarrier,meid,totalStorageSpaceInBytes,freeStorageSpaceInBytes,managed
DeviceName,partnerReportedThreatState,deviceConfigurationStates,deviceCategory,deviceCompliancePolicyStates" specified by
the ValidateSet attribute. Supply an argument that is in the set and then try the command again.
At line:1 char:38
+ $a = Get-IntuneManagedDevice -Select id,ethernetMacAddress | Get-MSGr ...
+                                      ~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : InvalidData: (:) [Get-DeviceManagement_ManagedDevices], ParameterBindingValidationException
    + FullyQualifiedErrorId : ParameterArgumentValidationError,Microsoft.Intune.PowerShellGraphSDK.PowerShellCmdlets.Get_De
   viceManagement_ManagedDevices

I tried your example of
$DeviceID = <some Device's ID>
$URI = [uri]::EscapeUriString("https://graph.microsoft.com/beta/deviceManagement/manageddevices('$deviceid')?`$select=UserPrincipalName,emailAddress,deviceName,model,manufacturer,serialNumber,id,lastSyncDateTime,ethernetMacAddress&_=1576515130721")
$dev = Invoke-MSGraphRequest -url $uri -HttpMethod get

I get the error message: Parsing OData Select and Expand failed: Could not find a property named 'ethernetMacAddress' on type 'microsoft.graph.managedDevice'.

Are you sure this property exists?

I have hit this same issue today.  Example:
$intuneDevices = Get-IntuneManagedDevice  | Get-MSGraphAllPages
$intuneDevices[0]
Returns an object with property (see below), but I receive the same error as @clickbangdead when using the select parameter with the property included.

Possibly that the set validation needs to be updated with the new property?

Hello, I am also experiencing this issue.

hi, do we have any update of this case?

still same issue here


Add openai custom api host

I have identified that the current implementation does not adequately handle errors resulting from incorrect or missing host inputs. This could lead to unexpected behavior or crashes. However, I have not yet implemented a solution for this issue in this PR.

Hi, thanks a lot for your contribution! Did you find a case where the error handling issue you mention also affects a "normal" perplexity and openai endpoints? Or is it only for your third party feature?

Sorry for the confusion caused by some variables. I will submit another PR. It won't take effect when the Custom API Host is not provided. The customization here is for OpenAI's host.

Review the new commit

Awesome, thanks for you PR!


OSLC 3.0 Core Compatibility

As a long term goal, it would be nice to achieve also OSLC Core 3.0 compatibility.

Hi @muggy83.
I have been reading and reviewing the differences between both specifications, in my point of view there are three main topics that should be taken in account to add support for OSLC v3 on PyOSLC.

OpenIDConnect.- The recommendation of OSLC v3 is to use OIDC to add the security layer to the endpoints, this requires the implementation of OAuth 2.0 since OIDC work upon OAuth 2.0, now we are working with OAuth 1.0a for the interaction with other platforms.
LDPPaging.- Since OSLC v3 works on LDP it requires to use this approach for managing the responses on the query capabilities.
Previews.- For the previews of Delegated UI, there are some adjustments that should be applied to manage the new headers to retrieve the dialogs.

There are other topics but for now these seem like the most impactful on the development.
We can start analyzing the way to start working on this new topics.
Regards!

Hello there, just wanted to add a few remarks:

OIDC is likely to be no. 1 thing for companies to adopt.
LDP support is optional. Furthermore, LDP paging is not part of the W3C LDP spec and OSLC decided to adopt LDP without recommending LDP paging and keep OSLC paging instead. Discussion on the https://forum.open-services.net/ or https://lists.oasis-open-projects.org/g/oslc-op welcome. See https://oslc-op.github.io/oslc-specs/specs/core/oslc-core.html#usingResourcePaging for details.
We finally mentioned Root Services in the spec and registered a "well-known" URI /.well-known/oslc/rootservices.xml to allow server connection to be set up more easily. See https://oslc-op.github.io/oslc-specs/specs/core/discovery.html#wellKnown
Regarding the dialogs, the most important change in the spec is the strong recommendation to set up CSP and CORS correctly: https://oslc-op.github.io/oslc-specs/specs/core/oslc-core.html#resourceRepresentations and https://oslc-op.github.io/oslc-specs/specs/core/dialogs.html##embed_policy

Good luck and congrats on the release of the Python OSLC support!
Cheers,
Andrew
OSLC PGB co-chair

Thanks for the insights, this is very welcome!


Length of fixed_size_collection |> Take(n) is unkonwn

To my surprise, a transducer using Take does not know it's size, even if there are no Filter calls in the pipeline:
julia> 1:100 |> Take(10) |> length
ERROR: ArgumentError: Base.IteratorSize(ed) isa Union{Base.HasLength, Base.HasShape} must hold. Got
Base.IteratorSize(ed) => SizeUnknown()
Union{Base.HasLength, Base.HasShape} => Union{HasLength, HasShape}
Stacktrace:
 [1] macro expansion
   @ ~/.julia/packages/ArgCheck/5xEDR/src/checks.jl:243 [inlined]
 [2] length(ed::Transducers.Eduction{Transducers.Reduction{Take, Transducers.BottomRF{Transducers.Completing{typeof(BangBang.push!!)}}}, UnitRange{Int64}})
   @ Transducers ~/.julia/packages/Transducers/oLdU1/src/processes.jl:581
 [3] |>(x::Transducers.Eduction{Transducers.Reduction{Take, Transducers.BottomRF{Transducers.Completing{typeof(BangBang.push!!)}}}, UnitRange{Int64}}, f::typeof(length))
   @ Base ./operators.jl:858
 [4] top-level scope
   @ REPL[44]:1

Tangential thought:

Take(n)
Take n items from the input sequence.

This makes me think 1:3 |> Take(10) |> collect would error. Maybe I'd go with TakeAtMost(10) for the permissive version and TakeExactly(10) for the error.

Collect does not require to know the length so it would not error.


ci: Run tests against python3.7 instead of python3.6 and freeze pipenv

Hey @medikoo, sorry to bother again, but it seems like the required jobs checks are still on (I tried to create a new PR, I thought that maybe it won't refresh automatically on existing PRs), could you please check once again? Also, I will need another review from you (or maybe you could turn off requirement for at least 1 review so I can merge my own PRs without bothering you?) Thanks in advance 🙇

Hey @medikoo, sorry to bother again, but it seems like the required jobs checks are still on

Indeed, not sure why it didn't uncheck all first. I've ensured it's ok this time. I've also removed the requirement for review approvals. Thanks @pgrzesik !


span and range checking enhancements

New header span_util.h with span-related helpers:

make_span/make_cspan helps for converting vectors & arrays to spans when passing to templated functions.
spancpy, spanset, spanzero are memory-safe, range-checking substitutions for memcpy, memset when working with spans.

Add range check DASSERT to span access in debug mode. There were a couple spots where we were taking the address of an out-of-range value that needed to be adjusted to use a better idiom.
ParamValue: extra OIIO_DASSERT range check for get() method.
Simplify constructors by replacing &x[0] with using data(), which is always well-defined.
Fix typos in span.h comments

The range checking imposes no extra cost for release/optimized builds. But it might help us for CI analysis and any time we're debugging.

Does anybody have objections to this, or comments about it?
There are primarily two sets of changes in this PR:

Adding debug-mode-only range checking to the indexing operators for our span<> classes. Because it's debug only, there should be no release-build performance penalty for this, but it should make it easier to identify addressing range errors when debugging and when running the testsuite in CI, especially on the sanitizer and analysis runs.

Adding some new utility functions that do buffer copies with spans. We don't use these (yet), and are in a new header, so they can't possibly break anything that currently works. But I'd like to start using them where applicable, because it should give some additional protection for address range bugs.

Discussed in TSC, people like spans.


🚀🤖  Configure the Cloud Ops Bot to send API request to MoJ SNOW

User Story
As a Cloud Ops engineer
I want to use the Cloud Ops Bot to send API requests to the MoJ SNOW service.
So that I can test the new API created in SNOW
Value / Purpose
By completing this initial functionality testing, we can further develop the Cloud Ops Bot to raise incidents from Slack.
Useful Contacts
Cloud Ops Team
Additional Information
Does anyone have a link to the SNOW API documentation? @wanieldilson ?
Definition of Done
Cloud Ops Bot sends API requests to SNOW API to confirm connectivity and basic functionality.
Example - [ ] Documentation has been written / updated

[ ] README has been updated
[ ] User docs have been updated
[ ] Another team member has reviewed
[ ] Tests are green

New PR which makes the SNOW URL a parameter.

https://github.com/ministryofjustice/cloud-operations-slack-bot/issues/95 Raised to capture changing the SNOW URL secret in future.


Test that single validator node doesn't shut down in the absence of other validator nodes.

Verify that shuting down all nodes but one, allows for resuming it later while maintaining running node available for queries.
Setup:

Start 5-node validator network.

Query still-running node's HTTP endpoints.
Verify that it still answers queries.
Restart previously stopped nodes and wait for them to synchronize.
Verify that consensus resumes building blocks.

Closes https://github.com/casper-ecosystem/Governance/issues/96

bors merge


[MRTK3] Changing SimulatedHMD to public from internal

Describe the problem
It's just a keyword change, but this change will allow me to create a custom input simulator which I can utilize for Android and iOS mobile. With thischange, I have paired up an mobile phone's rotation and position based on unity canvas buttons. I could also pair them up with the device sensors for easier control.
Describe the solution you'd like
Simply changing SimulatedHMD to public instead of internal
Describe alternatives you've considered
For now I'm using the cloned repository. But it'll be nice for this to be changed in the source itself.
Additional context
It will also be nice to switch SimulatedController to public instead of internal
I have future plans to make use of a simulated controller using raspberry pi and this is the way for it i think.

@akash14darshan feel free to open a PR!

@akash14darshan, if this is still an issue, please feel free to open a PR (and reopen this issue). Otherwise closing as stale.
Your input sim idea sounds awesome! The XRI folks have actually designed a new input sim themselves with the new 2.3 update that might be worth taking a look at.


Add fields to support overrides in file decorations serving data [phab:D1325]

Created by schroederc at 2017-02-10 00:16:51:

zarko commented inline (context lost during migration) at 2017-02-10 01:09:47:
kind is deprecated (they're always DEFINITION; this was from a previous design before MarkedSource).

zarko commented inline (context lost during migration) at 2017-02-10 01:09:47:
I think this is the ticket of the node that would appear as the target of a Decoration.

zarko commented inline (context lost during migration) at 2017-02-10 01:09:47:
Given the above, I think that this is the node that's being overridden/extended (e.g,, Object if ticket were String).

zarko commented inline (context lost during migration) at 2017-02-10 01:09:47:
Maybe document this as the MarkedSource for the node being overridden/extended.

zarko commented inline (context lost during migration) at 2017-02-10 01:09:47:
Kind can go away.

zarko wrote at 2017-02-10 01:09:47:
Some questions about documentation and field names.

schroederc wrote at 2017-02-10 01:33:47:
Rename fields

schroederc commented inline (context lost during migration) at 2017-02-10 01:33:56:
I meant for them to be the reverse of your description.  Is this better?

schroederc wrote at 2017-02-10 01:35:15:
Remove stray comment

zarko commented inline (context lost during migration) at 2017-02-10 01:36:38:
Leftover comment?

zarko commented inline (context lost during migration) at 2017-02-10 01:36:38:
node whose ticket is overridden?

zarko wrote at 2017-02-10 01:36:38:
Yes, thanks.

zarko accepted this revision at 2017-02-10 01:36:38

schroederc wrote at 2017-02-10 01:52:53:
Fix comments

schroederc closed via commit da4838d1653a2df9a5a662b71ffbd22ce26743d7 at 2017-02-10 01:53:11


Adding debug mode to build

To debug the C/C++-calls and run in gdb a debug mode would be very handsome.
Something like:
if flag(debug)
  if os(linux)
     ghc-options: -Wall -g -rtsopts -dcore-lint -debug -O0
     cc-options: -g -O0 -fsanitize=address -fno-omit-frame-pointer
  if os(darwin)
    ....
  if os(windows)
   ...

Sounds good to me!

As far as I can see, there are no "free" variables in cabal. Therefore this decision tree has to be repeated for each module.
Does anyone know any trick? Otherwise I will create a PR with copy and paste-sections.

Try this https://cabal.readthedocs.io/en/latest/cabal-package.html#common-stanzas

@jpwidera Is the issue resolved with compiler flags merged?

Yes! :)


Speed up start up and reload times

This is a port of https://github.com/sbt/sbt/pull/3019 to sbt 1.0. To see the previous discussion, please have a look at that PR.
These changes are optimizations that affect the following sbt subsytems:

Sbt parser.
Plugin discovery.

A concise explanation for every change is provided in the commit message.  In
short, this PR makes sure that we don't load the whole Scala reflect universe
to check whether a plugin has a autoImport member or to parse a sbt file.
In informal measurements, these changes affect around ~12-14% of the whole
startup time. These are the results before:
jvican in /data/rw/code/scala/scala
> $ time sbt "exit"
[info] Loading global plugins from /home/jvican/.sbt/0.13/plugins
[info] Loading project definition from /data/rw/code/scala/scala/project/project
[info] Loading project definition from /data/rw/code/scala/scala/project
[info] Resolving key references (11310 settings) ...
[info] *** Welcome to the sbt build definition for Scala! ***
[info] Check README.md for more information.
sbt -mem 2048 "exit"  29.66s user 0.56s system 297% cpu 10.171 total

jvican in /data/rw/code/scala/scala
> $ time sbt "exit"
[info] Loading global plugins from /home/jvican/.sbt/0.13/plugins
[info] Loading project definition from /data/rw/code/scala/scala/project/project
[info] Loading project definition from /data/rw/code/scala/scala/project
[info] Resolving key references (11310 settings) ...
[info] *** Welcome to the sbt build definition for Scala! ***
[info] Check README.md for more information.
sbt -mem 2048 "exit"  30.52s user 0.46s system 299% cpu 10.337 total

jvican in /data/rw/code/scala/scala
> $ time sbt "exit"
[info] Loading global plugins from /home/jvican/.sbt/0.13/plugins
[info] Loading project definition from /data/rw/code/scala/scala/project/project
[info] Loading project definition from /data/rw/code/scala/scala/project
[info] Resolving key references (11310 settings) ...
[info] *** Welcome to the sbt build definition for Scala! ***
[info] Check README.md for more information.
sbt -mem 2048 "exit"  28.85s user 0.43s system 293% cpu 9.988 total

And after:
jvican in /data/rw/code/scala/scala
> $ time sbt "exit"
[warn] Executing in batch mode.
[warn]   For better performance, hit [ENTER] to switch to interactive mode, or
[warn]   consider launching sbt without any commands, or explicitly passing 'shell'
[info] Loading global plugins from /home/jvican/.sbt/0.13/plugins
[info] Loading project definition from /data/rw/code/scala/scala/project/project
[info] Loading project definition from /data/rw/code/scala/scala/project
[info] Resolving key references (11355 settings) ...
[info] *** Welcome to the sbt build definition for Scala! ***
[info] Check README.md for more information.
sbt -mem 2048 "exit"  24.83s user 0.51s system 286% cpu 8.831 total

jvican in /data/rw/code/scala/scala
> $ time sbt "exit"
[warn] Executing in batch mode.
[warn]   For better performance, hit [ENTER] to switch to interactive mode, or
[warn]   consider launching sbt without any commands, or explicitly passing 'shell'
[info] Loading global plugins from /home/jvican/.sbt/0.13/plugins
[info] Loading project definition from /data/rw/code/scala/scala/project/project
[info] Loading project definition from /data/rw/code/scala/scala/project
[info] Resolving key references (11355 settings) ...
[info] *** Welcome to the sbt build definition for Scala! ***
[info] Check README.md for more information.
sbt -mem 2048 "exit"  24.46s user 0.46s system 287% cpu 8.674 total

jvican in /data/rw/code/scala/scala
> $ time sbt "exit"
[warn] Executing in batch mode.
[warn]   For better performance, hit [ENTER] to switch to interactive mode, or
[warn]   consider launching sbt without any commands, or explicitly passing 'shell'
[info] Loading global plugins from /home/jvican/.sbt/0.13/plugins
[info] Loading project definition from /data/rw/code/scala/scala/project/project
[info] Loading project definition from /data/rw/code/scala/scala/project
[info] Resolving key references (11355 settings) ...
[info] *** Welcome to the sbt build definition for Scala! ***
[info] Check README.md for more information.
sbt -mem 2048 "exit"  25.12s user 0.45s system 290% cpu 8.802 total

The error https://travis-ci.org/sbt/sbt/jobs/223197672 is reproducible from latest master locally, which hints this PR is not causing it.

The error has been fixed and a more stable implementation of the autoImport detector has been added.
I think GitHub has a bug in the UI and is showing the commits in an incorrect order. The CI has kicked in for the last one.

The correct order is here. https://github.com/scalacenter/sbt/commits/speed-up-startup-time-1.0

@eed3si9n @dwijnand This is ready for review again. Please, can you have a look?


create_lamigo_service plugin

Create a create_lamigo_service  action plugin that does the following:

Creates a unit in /etc/systemd/system/ named lamigo-<FS_NAME>-<MDT_NAME>.service that runs: lamigo -a <MIN_AGE> -m <FS_NAME>-<MDT_NAME> -u <CHANGELOG_USER_NAME> -t <COLD_POOL> -s <HOT_POOL> <MNT_PNT>.

A start action should be to mount the filesystem if it is not mounted already.

Mount of lustre client can be it's own service start, I'll take care of then in EX-741

@utopiabound I.E. systemd mount unit?


Merge changes from the 3.1 branch to 3.2

Merge changes from the 3.-maintenance branch to the 3.2 branch

We actually interested only in the last commits plus some docs update,

may i ask why the cpu profiling support change got reverted? i was looking forward to using that.

It is available in the 3.2 branch. It was originally submitted by a mistake to the 3.1 branch and then reverted. The revert commit shows up on the merge, but if you take a look at the diff, no CPU profiling changes are reverted.

thanks for the clarification!!


Link git submodules

pytti-core setup currently recommends installing everything in pytti-notebook as root directory. We can do this for the users.

Add git submodules: AdaBins, GMA, CLIP, taming-transformers, pytti-core
Add folders with .gitkeep: ./pretrained, ./images_out, ./videos

Desired structure:
 ├── pytti-notebook
 │   ├── AdaBins
 │   ├── CLIP
 │   ├── GMA
 │   ├── images_out
 │   ├── pretrained
 │   ├── pytti-core
 │   ├── taming-transformers
 │   └── videos

fun fact: we can also conda install git. so setup would then be:

install anaconda
load and activate conda env
git clone recursive pytti-notebook
pip install research code

After conda installed, I think those last three steps could be scripted pretty easily.


Create new screenView context (2-0-0) with no minProperties requirement

This is so we can record screen view events in e.g. Snowplow mobile trackers and send the relevant information in a separate screen context.

Would make sense to remove all the properties in this event so that they are all sent in the separate screen context


Add support for non-JSON payloads

I have a use case where I need to produce a JWT for non-JSON data and it would be nice to be able to use this library to achieve that.

Note that tokens generated with this would not be standardscompliant and you basically removed all features of JWT.
If all you want to do is sign and verify random payloads you can skip the jwt parts and directly use the algorithms. They all follow the same interface and are binarysafe, so you can pass whatever your payload is.

They wouldn't be valid JWT - that is true.  They would be valid JWS in Compact Serialization, which is what I am trying to achieve.   Granted, that's taking your library a bit off the original goal, but I think it's perfectly reasonable.  (and I've done this in a fork if you want to see)

Changes are in my branch at https://github.com/lrosenthol/jws-jwt-cpp

Did you verify the token in some other library ?
Because if I understood the spec correctly, then the payload is included in the signature (in detached mode) but not in the token.

If you think about it as a detached JW Signature and not a token, yes, it works fine - provided that both sides have apriori agreement on what the payload should be.  In my case, it's an image that goes along with the JWS bytes.

This seems to be out of the scope for the project, the majority of JWT use the JWS JSON Serialization.

This seems to be out of the scope for the project, the majority of JWT use the JWS JSON Serialization.


Illegal memory access in load_textures

I am trying to load an obj file into TriangleMesh with with_vt=True. The mesh contains multiple textures. While the first texture image gets loaded into GPU correctly , I am getting following error for subsequent texture images. The error occurs at this line
RuntimeError: CUDA error: an illegal memory access was encountered

Specs:
pytorch                   1.3.1           py3.6_cuda10.1.243_cudnn7.6.3_0    pytorch
cudatoolkit               10.1.243             h6bb024c_0

GPU: Titan Xp
Driver Version: 440.33.01
CUDA Version: 10.2,  V10.2.89

Me too, I encountered the same problem when loading .obj file with textures. It seems because this line:
textures = load_textures_cuda.load_textures(image, faces, textures, is_update)
If I replace it with:
textures = torch.ones([faces.size(0), texture_res ** 2, 3])
Then it will not cause CUDA error, working normally like when the parameter load_texture is set False in load_obj function.
Because the function load_textures_cuda in module soft_renderer.cuda is written in .cpp and .cu file, and I don't know how to make my code call it correctly, so my debugging is interrupted. Anyone can help me?

Hi, thank you for using Kaolin! The library went through a major rewrite with the version 0.9.0, and this issue is no longer current.
Please refer to our latest documentation:
https://kaolin.readthedocs.io/en/latest/


[Bug] Crash upon opening inventory with a potion effect.

Hi, I splashed a fire resistance potion in the nether while using EMI + JEI and now upon opening my inventory, my game crashes. I believe the issue has to do with this mixin from the stacktrace: https://github.com/emilyploszaj/emi/blob/93e0d4f33f61299255947656367bd0429baa50a6/xplat/src/main/java/dev/emi/emi/mixin/AbstractInventoryScreenMixin.java#L75
My logs are: https://mclo.gs/Nz874EW

My mixin is in the log but it's not causing the issue, this is on porting lib

Ah sorry!! I just realized and was going to close this issue. The issue was actually https://github.com/Fabricators-of-Create/Porting-Lib/issues/75. Thank you! This mod is awesome


Add Support for chatglm2

1.edit rotary_embedding to support chatglm2 and use vllm's multi-query attention for chatglm2
2. I have test the result
a. Inference result alignment.
b .Performance result: greedy mode when input_token_name=250, max_output_token=50, greedy mod; torch Throughput 22samples/s ; vllm Throughput 37.4samples/s

Inference result alignment.
2 .Performance result: when input_token_name=250, max_output_token=50; torch Throughput 22samples/s ; vllm Throughput 37.4samples/s

1.edit rotary_embedding to support chatglm2 and use vllm's multi-query attention for chatglm2 2. I have test the result a. Inference result alignment. b .Performance result: greedy mode when input_token_name=250, max_output_token=50, greedy mod; torch Throughput 22samples/s ; vllm Throughput 37.4samples/s

Found (prompt)context decoding process aligned but tokens'generation not, possibly due to the cuda kernel.

The key problem is the cos_sin_cache is not aligned with chatglm2, because chatglm2 use mixed-precision to compute it. I will provide the corresponding code later. Maybe you can load it from chatglm2.

Again, thank you for your contribution! We merged https://github.com/vllm-project/vllm/pull/1261 and the current main branch supports ChatGLM now. Let us know if the main branch does not look good and feel free to propose any changes!


[POC] - bridge to base link

Description
This PR adds a button to the sidebar as a POC which links to squid router with pre-filled chains and token.

@Atmosfearful if you can take a look at this before I merge for a final sign off that would be great.


Backend Full run not working

Hey, i just compiled the last version of the backend and the full run update doesn't work (it doesn't also with older version).
23:16:31 [INFO] PICSChanges: Doing a full run on all apps and packages in the database.
23:16:31 [INFO] PICSChanges: Requesting info for 0 apps and 0 packages

Valve broke it.

Valve added a limit on that, so it no longer works.

Sorry to post in a closed bug, is there any workaround for populating an empty backend, or does someone have a SQL dump of a fully populated database that I could use?


USF 1629 - Add coupon container in cart

Please always provide the GitHub issue(s) your PR is for, as well as test URLs where your change can be observed (before and after):
Fix Ticket: https://jira.corp.adobe.com/browse/USF-1629
Test URLs:

Before: https://develop--aem-boilerplate-commerce--hlxsites.aem.live/
After: https://cart-coupons-two--aem-boilerplate-commerce--hlxsites.aem.live/

@cctina516 QA Passed.

@cctina516 please review the checkout coupons experience: as the removal of coupons are not being applied to the order summary totals.

@ericeoeur

Checkout coupon experience. As we discussed with @jcalcaben , it looks like an issue with the EstimateShopping container that checkout dropin uses, which doesn't emit an estimate/shipping event when the coupon is applied to the cart. Will create a bug ticket for it.
The "estimated tax and shipping" slot. It doesn't seem to do with the coupon container, but with Estimated shipping and tax being updated with any re-render.

Initial state with no coupon applied:

Increase the quantity to 2, still no coupon applied:

We can see the Estimated shipping and tax being activated regardless of any coupon actions. Therefore the issue is not with the coupon container, Will create a ticket for it.

checking...

Thanks @cctina516 QA approved. Will merge.


maxImageCacheCount does not work on iOS devices

OpenSeaDragon version 2.3.1
Total canvas memory use exceeds the maximum limit (224 MB).
TypeError: null is not an object (evaluating 'this._renderedContext.drawImage')

Above error is shown when running on XCode Simulator iPad 9th generation iOS 15.2.
I changed maxImageCacheCount to 60 from default 200 to prevent canvas memory from exceeding limit.
However, I still see over 128 canvas image in Graphics Tab in Web Inspector.
Is it caused by same problem as link below?
https://pqina.nl/blog/total-canvas-memory-use-exceeds-the-maximum-limit/

Yes, that's a likely culprit! Releasing our canvases like they suggest in that post sounds like a good plan.
Would you be up for working up a fix? If so, I'm happy to help point you in the right direction. If not, hopefully someone will take this on.


Bug(incorrect matching of data)

We recently discovered a case where the variants were told to be mismatched by the UI which was also confirmed by using the genotype check CLI command.
However, when we investigated these variants further we found that:

The sample was Homozygous (0 reads supporting REF) for ALT
The sample was Homozygous for ALT in both MAF and Sequence data
The positions of concern are on different chromosomes and are not a specific base.

This is very concerning - as it would seem that either the data uploaded to Genotype is not what we believe it to be OR we are doing an incorrect match for these Homozygous calls.

Issue caused by demux using wrong sample sheet, and hence reads from multiple samples. Hence genotype is functioning correctly. Overwrite the previous data using cg upload genotypes <case_id> -f


Improve documentation

Our documentation needs a boost!

[x] Community profile
[x] Hands-on tutorial
[x] How to run headless
[x] How to chose GPU when running headless #116
[x] How to create your own scenarios with our tools #110
[x] Running CARLA server in-editor #143
[ ] Software design #145
[ ] How many cars and pedestrians can you add? #171
[ ] More FAQ/Troubleshooting (get from questions list and add to documentation)
[x] Better road-map
[ ] Suggestions?

All contributions are appreciated ;)
Submit your commits/pull-requests to the documentation branch.

New documentation can be found here: https://carla.readthedocs.io/en/latest
Thanks to @sergi-e for his amazing work!


Cloud Configuration Pentests

The link Cloud Providers (PDF) is broken

thanks @bijan-grid  - will take a look at!

@bijan-grid this has been updated


ModuleNotFoundError: No module named 'androguard.core.bytecodes'

Please help  me to resolve this

Given that the issue refers to a rather old version of the project and the project is substantially different nowadays, I will be closing this.
Please feel free to reopen it in case it still stands.


Owncloud Deployer: Terms & Conditions Checkbox

By default the Submit button is not disabled when the user leaves the checkbox unchecked

It only gets disabled when the user checks the box then unchecks  it again

Yes, can confirm that,
and if you click on SUBMIT, it tries to send the form


Small updates to docs/users.md to reflect current API

It seems like email is not exposed anymore.
Also auth_token is available only on initial post to /users/ (during user registration).
Not sure of docs format --- how do you distinguish between URL params, query string params, and POST data?

Closes #172 for release #167

@ivanistheone thanks for contributing!
The docs' format matches Github's api documentation. Distinguishing url parameters is pretty straight forward because they're always contained in the provided resource url. For example users/:id/follow. They're not included in the parameter list ( unless its also valid there ).
Query string vs including data in the body of the request is also pretty straight forward because of the limitations of GET and POST methods. For example, if there is a documented parameter list for a GET request, it must be included in the query string of the request. Depending on the content type, for POST requests either can be used by the client.
Your pull request looks great! I'd like to make just a few updates so that it more closely matches the Github documentation format.  I'll add comments!

@ivanistheone this is all set to merge!
Do you want me to wait so you can update the urls with a leading slash or is it cool if I merge through?

Go for it!


Visually distinguish versioned docs

There will be two kpt sites co-existing for a while:

https://googlecontainertools.github.io/kpt  for master (0.39)
https://kpt.dev for next (1.0+)

Both sites should make it very obvious what kpt version they're targeting. If a user does a Google search and lands in one of these, there should be a prominent way to quickly figure this out what version the doc is talking about.
Related: Figure out a docs deprecation plan, considering SEO.

in backlog grooming, moving to M3 to focus on functionality

Should consider having kpt.dev use a different theme as well, currently it uses the same color palette as existing site. I'm personally a fan of dark themes (you can use an extension to see how it would look), but don't have a strong opinion.

I think a different theme is a good idea.  Dark is good but also since this is a "Google Cloud Tools" production we might want to have something that aligns more with https://cloud.google.com/docs
Either way this is probably something we can save for M4.

Yes, there are multiple aspects to this issue which can be separated:

[ ] New site should show the CLI version (1.0+) as docs are versioned (Visible on every page)
[ ] Old site should show it's targeting pre 1.0 version. Do not need to make major surgery to the old site, probably a banner  on every page suffices.
[ ] In addition, using a different theme to as an additional differentiator.
[ ] After the release, new site should be indexed instead of the old one and need to optimize SEO

Sent a request to temporarily remove site from index and merged a change that should indicate the docs are for an in-development version of kpt.


GenomePrep databases seem to be offline

This means no chip clusters or bad SNPs detection, as they cannot be fetched.

Both files are mirrored on Zenodo, so it could work as a backup source (or primary if the site never comes back):
https://zenodo.org/records/5047472/files/the_list.tsv.gz
https://zenodo.org/records/5047472/files/badalleles.tsv.gz


CreateTaskMachine API for spawning tasks

No longer needed (this can be done as part of user code). Closing the PR.


Merge the multi-source work from @k0nserv / @hjanuschka

Alright, this takes the advice from https://github.com/danger/danger/pull/299#issuecomment-235624279
And is manually merging in the work necessary to allow others work on things like bitbucket or #382  👍

          2 Errors

:no_entry_sign:
Please include a CHANGELOG entry.
You can find it at CHANGELOG.md.

:no_entry_sign:
The core plugins are not at 100% doc'd - see:
```
Files:           4
Modules:         1 (    0 undocumented)
Classes:         4 (    0 undocumented)
Constants:       0 (    0 undocumented)
Attributes:      0 (    0 undocumented)
Methods:        35 (    1 undocumented)
 97.50% documented
Undocumented Objects:
(in file: lib/danger/danger_core/plugins/dangerfile_github_plugin.rb)
Danger::DangerfileGitHubPlugin.new
```

  Generated by :no_entry_sign: danger

@orta You wanna pick 7f54b24 too it contains tests for CommentsHelper or I can do it and push it to this branch

It's intended to be squashed into 73754c2

I'm not really a re-writing history kind of person

👯

As in your are against it on a ideological basis?

Nah, I just prefer to see the history as immutable - it's mostly laziness, so I've never bothered to learn

Okay do you want me to do it then? I prefer to see history as immutable after it reaches stables branches, but not in feature branches.

Sure, you're welcome to it, 👍

All CI should pass now. If it does give me a shout when you are ready to merge and I'll rebase the branch and clean up the history a bit before :)

Okay also improved the test coverage of CommentsHelper so that at least all methods are tested. I think there's some extra coverage in the tests for the github request source as well.

Awesome 💃

This looks good to me - the windows CIs seem to have been queued for a day - so I'm OK with them not being green

Cool, I'll clean up the history and then merge

All except windows is green, merging this


Improve translation of title and first paragraph

Replace "declaración" by "sentencia" as translation of "statement".

[x] I have read freeCodeCamp's contribution guidelines.
[x] My pull request has a descriptive title (not a vague title like Update index.md)
[x] My pull request targets the master branch of freeCodeCamp.
[x] None of my changes are plagiarized from another source without proper attribution.
[x] My article does not contain shortened URLs or affiliate links.

If your pull request closes a GitHub issue, replace the XXXXX below with the issue number.
Closes #XXXXX

Thank you for your contribution to the page! 👍
We're happy to accept these changes, and look forward to future contributions. 🎉


overload new/delete operator with jemalloc

Currently we are not actually using jemalloc, because the new operator is not overloaded to use jemalloc. I would like to fix this, is there a global header/source that is included by all as long as we overload them somewhere that is linked in libpeloton, jemalloc would work

I also didn't see where do we include <jemalloc/jemalloc.h>, jemalloc is also not in the link flags, are we really using jemalloc?

No, we are currently not using jemalloc. Instead of overloading those operators, can you alter the linker flags to use -ljemalloc by adding the autoconf check (jemalloc.m4) ? Also, add it to the installed package list.
https://stackoverflow.com/questions/9521226/c-stl-with-jemalloc
https://github.com/chriso/bitset/blob/master/configure.ac#L27
https://github.com/chriso/bitset/blob/master/m4/jemalloc.m4
https://github.com/cmu-db/peloton/blob/master/scripts/installation/packages.sh

@jarulraj -ljemalloc alone will not work, the suggested way to make all new/delete and std::allocator to use jemalloc is to overload these operators (it is suggested in the first link you posted)

I think I've set the link flag, haven't I?

@yingjunwu Yes I saw a -ljemalloc-dev, but that alone would not work, we are still using malloc

simple benchmark:
#include <vector>
#include <boost/thread.hpp>
#include <jemalloc/jemalloc.h>
#include <iostream>

/* uncomment to replace with jemalloc
void* operator new(std::size_t sz) {
    return malloc(sz);
}
void operator delete(void* ptr) noexcept
{
    free(ptr);
}
*/

int main() {
    std::vector<boost::thread> threads;

    for (int i = 0; i < 12; i++) {
        threads.push_back(boost::thread([]() {
            for (int j = 0; j < 1024*1024; j++) {
                int *number = new int;
                *number -= 1;
                delete number;
                std::vector<int> vec(1024);
                vec[1023] = -1;
            }
        }));
    }

    for (auto &thread : threads) {
        thread.join();
    }

    return 0;
}

Without those overloaded operators
g++ -O3 -std=c++11 -o test test_jemalloc.cpp -lboost_system -lboost_thread -ljemalloc -lpthread
time ./test
./test  3.69s user 0.01s system 914% cpu 0.404 total

With overloaded operators
g++ -O3 -std=c++11 -o test test_jemalloc.cpp -lboost_system -lboost_thread -ljemalloc -lpthread
time ./test
./test  0.01s user 0.00s system 166% cpu 0.005 total

It's on the MemSQL machine

I think you can just use LD_PRELOAD. It's not very cool to change the source code for using jemalloc.

https://github.com/jemalloc/jemalloc/wiki/Getting-Started

I think that the issue is the order of the link flags. Add -ljemalloc before another other libraries, so that they don't load in glibc malloc, and see if that helps.
http://www.canonware.com/pipermail/jemalloc-discuss/2013-July/000619.html

you can also learn from silo: https://github.com/stephentu/silo. They support jemalloc, tcmalloc, hoard etc. They did not change the source code.

They did
https://github.com/stephentu/silo/blob/master/memory.cc

we can talk offline.

Finished this in 1c16a21810.


feat(_scripts): build every tagged version of workflow docs

Since gutenberg uses rsync to pull in the projects for building the docs, the tags should be available for building.
ping @slack; Would this be the right way to do it?
closes #351

punting

ping @slack, you mentioned you had a better idea

ping

This should be fine, alternatives aren't worth the effort!


Import documentation for @lukso/lsp-utils

What does this PR introduce?
This PR adds a new sidebar in the tools for the repository @lukso/lsp-smart-contracts-utils

Let's wait for the package to be released first.
You can also add a page in this section:
https://docs.lukso.tech/tools/getting-started


Interactive Issue Resolution Workflow

I propose to help build a feature that will help maintainers create an interactive issue resolution workflow directly within the Knip CLI tool. This workflow will guide users through addressing identified issues by providing contextual information, recommended actions, and integration with version control.

Sure I can help build it while reaching to the core team occasionally to ensure we build value around it.
Also, I do document stuff (German and English) thanks and let me know what progress and input is needed.

I need to rethink file management, dependencies and such, then will come with a solid and flexible approach. Let's keep the issue open as I use and study the tools too

Going to close this issue due to inactivity. Feel free to discuss further or open a new one.


New version...

Do I need to install the new version after deleting the old version?
http://i.imgur.com/iRQb0na.png
http://i.imgur.com/J83vzJE.png

Yeah, had the same issue, just recently closed it.

Installing on top of the old version gives error? Like error while parsing the package or sth? I'll try to check that signature tomorrow.


Push down filters below Unnest in sub queries

Is your feature request related to a problem or challenge?
Currently where clauses on a sub query are not pushed below Unnest expressions. In conjunction with some other limitations, this makes composing queries with unnest() not ideal.
In 39.0.0 cli:
> CREATE TABLE IF NOT EXISTS v AS VALUES(1,[1,2,3]),(2,[3,4,5]);
0 row(s) fetched.
Elapsed 0.008 seconds.

We get equal results with or w/o a subquery:
> select unnest(column2) from v where column1=2;
+-------------------+
| unnest(v.column2) |
+-------------------+
| 3                 |
| 4                 |
| 5                 |
+-------------------+
3 row(s) fetched.
Elapsed 0.011 seconds.

> select "unnest(v.column2)" from (select unnest(column2), column1 from v) where column1=2;
+-------------------+
| unnest(v.column2) |
+-------------------+
| 3                 |
| 4                 |
| 5                 |
+-------------------+
3 row(s) fetched.
Elapsed 0.015 seconds.

Filter and FilterExec are above projections with unnest, instead of below.
Ok:
> explain select unnest(column2) from v where column1=2;
+---------------+------------------------------------------------------------------------+
| plan_type     | plan                                                                   |
+---------------+------------------------------------------------------------------------+
| logical_plan  | Unnest: lists[unnest(v.column2)] structs[]                             |
|               |   Projection: v.column2 AS unnest(v.column2)                           |
|               |     Filter: v.column1 = Int64(2)                                       |
|               |       TableScan: v projection=[column1, column2]                       |
| physical_plan | UnnestExec                                                             |
|               |   RepartitionExec: partitioning=RoundRobinBatch(8), input_partitions=1 |
|               |     ProjectionExec: expr=[column2@1 as unnest(v.column2)]              |
|               |       CoalesceBatchesExec: target_batch_size=8192                      |
|               |         FilterExec: column1@0 = 2                                      |
|               |           MemoryExec: partitions=1, partition_sizes=[1]                |
|               |                                                                        |
+---------------+------------------------------------------------------------------------+

Problematic:
> explain select "unnest(v.column2)" from (select unnest(column2), column1 from v) where column1=2;
+---------------+---------------------------------------------------------------------------------------+
| plan_type     | plan                                                                                |
+---------------+---------------------------------------------------------------------------------------+
| logical_plan  | Projection: unnest(v.column2)                                                         |
|               |   Filter: v.column1 = Int64(2)                                                        |
|               |     Unnest: lists[unnest(v.column2)] structs[]                                        |
|               |       Projection: v.column2 AS unnest(v.column2), v.column1                           |
|               |         TableScan: v projection=[column1, column2]                                    |
| physical_plan | ProjectionExec: expr=[unnest(v.column2)@0 as unnest(v.column2)]                       |
|               |   CoalesceBatchesExec: target_batch_size=8192                                         |
|               |     FilterExec: column1@1 = 2                                                         |
|               |       RepartitionExec: partitioning=RoundRobinBatch(8), input_partitions=1            |
|               |         UnnestExec                                                                    |
|               |           ProjectionExec: expr=[column2@1 as unnest(v.column2), column1@0 as column1] |
|               |             MemoryExec: partitions=1, partition_sizes=[1]                             |
|               |                                                                                |
+---------------+---------------------------------------------------------------------------------------+

Describe the solution you'd like
FilterExec is pushed down to tables that support them.
Describe alternatives you've considered
A workaround is trying to not use subqueries. However, then one cannot unest structs with multiple lists, or unnest a column more than once. Those are also unsupported.
> select unnest(unnest([[1],[1,2],[1,2,3]]));
type_coercion
caused by
This feature is not implemented: Unnest should be rewritten to LogicalPlan::Unnest before type coercion

> select unnest(column2) as x, unnest(column2) as y from v;
Error during planning: Projections require unique expression names but the expression "v.column2 AS unnest(v.column2)" at position 0 and "v.column2 AS unnest(v.column2)" at position 1 have the same name. Consider aliasing ("AS") one of them.

(note: a real case would be something like select unnest(named_struct)['foo'] as foo, unnest(named_struct)['bar'] as bar).
Additional context
Maybe related: #5364

For this kind of query select unnest(unnest([[1],[1,2],[1,2,3]]));, I think we can follow what Duckdb is.
Unnest with recursive, and max_depth
select unnest([[1],[1,2],[1,2,3]], recursive:= true);
or select unnest([[1],[1,2],[1,2,3]], max_depth := 2);

I think this is related https://github.com/apache/datafusion/issues/10660

Thanks for the pointers @duongcongtoai
I think the recursive and max_depth options can be a special workaround, but introduce a good amount of complexity. Fixing the push down makes single unnest composable in general.

@ahirner
I found an alternative query is to move filter into the query
select "unnest(v.column2)" from (select unnest(column2), column1 from v where column1=2);
query TT
explain select "unnest(v.column2)" from (select unnest(column2), column1 from v where column1=2);
----
logical_plan
01)Projection: unnest(v.column2)
02)--Unnest: lists[unnest(v.column2)] structs[]
03)----Projection: v.column2 AS unnest(v.column2), v.column1
04)------Filter: v.column1 = Int64(2)
05)--------TableScan: v projection=[column1, column2]
physical_plan
01)ProjectionExec: expr=[unnest(v.column2)@0 as unnest(v.column2)]
02)--UnnestExec
03)----RepartitionExec: partitioning=RoundRobinBatch(4), input_partitions=1
04)------ProjectionExec: expr=[column2@1 as unnest(v.column2), column1@0 as column1]
05)--------CoalesceBatchesExec: target_batch_size=8192
06)----------FilterExec: column1@0 = 2
07)------------MemoryExec: partitions=1, partition_sizes=[1]

Given that, it might not be neccessary to support pushing down outer filter

alternative query

Yes. The problem is that one can't move the filter to the inner query (views) or one shouldn't have to with other common sql constructs (sub queries, ctes).

I think we can rewrite Unnest plan in push down filter

PR to do so: https://github.com/apache/datafusion/pull/10974

I confirm that https://github.com/apache/datafusion/pull/10974/commits/1b396c449a1fe0d8d359cbfde65f762958d3e241 fixes our more involved case in practice.

Thanks for the swirt action.
I just found out this second, that the change introduces a new issue when filtering on an unnested struct column. Sorry for being late.
I'll file a new issue. The reproducer is less trivial as here.

Added issue that's quite minimal but maybe not entirely minimal: https://github.com/apache/datafusion/issues/10990

I confirm that 1b396c4 fixes our more involved case in practice.

Thank you @jayzhan211 for the quick response


Controller properties not being displayed in generated docs

In the example files for documenting a controller:
    /**
     * @property {Hash} _objects collection og objects that belongs to this map
     */
    this._objects = {};

but the output.js file doesn't contain this property.
Any advice for documenting controller properties or/and why the example doesn't work?

Could you please provide the whole code of the controller?

If you used @name for the parent docblock it will ignore all nested docblocks. Post your objects docblock.

@Zemke @reloaded this is the case on your example controller: https://github.com/allenhwkim/angular-jsdoc#example

It will work if you add  the "@memberof" annotation like so:
    /**
     * @memberof MapController
     * @property {Hash} _objects collection og objects that belongs to this map
     */
    this._objects = {};

The example seems to be missing it and that's basically it.
I've been documenting a quite complex AnguarJS/NodeJS/Browserify project for the last few days and any visibility problems went away when I realized that I need to add a proper "@memberof" to every entry, for example:
/**
 * @memberof module_name
 */

for top level entries
/**
 *@memberof module_name.DirectiveName
 */

for entries nested in a directive, which has a "@name DirectiveName" and "@memberof module_name"
It's a basic parent-child relation from what I figured. It's a bit annoying to have to add this to every inner part you want to document, but I guess writing a parser that would know where it currently is (inside or outside of a function/class, etc.) would be a lot of work.
Maybe the docs should have a short notice about it?


A little problem

I want to plot the loss function and psnr curve of the training process. What should I do? Painted like you

Just run tensorboard on the log directory
tensorboard --logdir <your directory>


feat(echo-pipeline): Make ECHO load only last epoch

copilot:all

Current dependencies on/for this PR:

main

PR #3586   👈

This comment was auto-generated by Graphite.


Changing color of your window borders and taskbar crashes VSCode

VSCode Version: 1.18.0
OS Version: Windows 8.1 Enterprise

Steps to Reproduce:

Open VSCode
Open Control Panel > Personalization > Color and Appearance
Set Hue/Saturation/Brightness to lowest values.
Slide Color intensity to highest value

VSCode crashes.
Reproduces without extensions: Yes

@howiempt which dialog are you in on Windows, can you share a screenshot?

Dialog below:

Basically just drag the Color intensity bakc and forth very quickly :+1:

@howiempt can you try to reproduce with our newer Windows build: https://az764295.vo.msecnd.net/exploration/ed422f909ea470f79e3e3e39c948803d9b764919/VSCode-win32-x64-1.22.0-exploration.zip

@bpasero unable to reproduce with that build, looks good!


Add foralls into Term language so that they can benefit from scoped variables

Problem: foralls are only allowed at the global level.
Solution: Add foralls to the Term language.
let f(xs: Integer[]) = {
   forall x: xs. x : Odd
}

Alternative: this could be forced into the global space with more hard quantified implications.
Downside: the time complexity goes way up due to an inability to localize searches. There would have to be not only type level substructure pattern matching, but also term level substructure pattern matching.
forall x: Odd. Odd => Even = x + 1
forall x: Odd. Odd => Even = x - 1
forall x: Even. Even => Odd = x + 1
forall x: Even. Even => Odd = x - 1

to apply these rules you would sometimes need to pattern match any (x + 1) or (x - 1) terms that are Even or Odd which gets out of hand quick. There is no limit to the time complexity here.

Closing in preference of named foralls.


Implement Drawer

Application should implement navigation drawer in all activities.
We should probably move from activities into fragments.

Maybe BottomNavigation instead of drawer?

@noiserr I made a poll

This was done in lastest commits


translate oasst error japanese

Hey there, not sure if its a configuration issue on my end but trying to create a japanese dataset and comes upto the end of the run and starts loading up all my vram, goes till it can fit anymore then dumps and starts again, not sure if its normal behavior?  should i just leave it
running command  python translate_oasst.py ja ja 500 20
screenshot of attached behavior

This is to be expected. We first take all messages of one source language (say, English) together and translate those in batches. Once done, that vRAM is freed and we proceed with the next source language (say, Spanish). You will see the translation models being loaded in console and brief releases of vRAM.
As long as you keep getting new checkpoint files, you are good.

Just to point out: the language code that OPUS uses for Japanese seems to be jap instead of jp.

Hey Eric, thanks for the reply! Originally i had used Jap as my code but i was running into this error, so i tried jp on the seccond pass to check but it was still doing it, as the dataset progresses is fine, the back and forth is on that last step of generation config, it does the first one then just goes up and down not seeming to process, i left it for half an hour and it was still just loading and unloading the ram in a loop it seemed? when it was loading during the run it would only push maybe 2gb or so, but the last one is pulling all 12gb and  additional ram and not progressing? hopefully i have explained that well enough as it just doesnt seem like it wants to move forward. could my batch be too high for my card size?
thank you for your help :)

Well it does them one language at a time and usually there is a large portion of "rare" languages that only have a few records and it needs to load and unload models for quite frequently but it should never stall. The empty arrays only result in case there is no translation possible, not even through English.
I will see if I can run Japenese some time this or next week to debug.

I do see the Japanese models are a bit larger than average - have you tried using a batch size of 10 instead of 20?

I am running "jap" now, seems like both "ja" and "jap" are Japanese. I see vRAM going up to 14.6GB but I am not seeing any errors yet, going strong on a 16GB GPU with 20 batch size. I will let it run and publish the datasets once finished.

so ive tried again overnight, when i left it, was running fine on batch = 10 kept my vram usage at 8gb, waking up this morning now its at a crawl and usage is pinged up again, seems like it has progressed but from a point it seems to be really really slow
heres another screenshot, i did note on your original post this would take a long time to produce, is that the case and i just need to wait for now?
with this run looking at the json files it seems to actually have data in there now so i guess it must be something on my end

Yeah 1-1.5s/it is pretty normal. I noticed when doing jap (almost done btw) that for EN-JAP it is significantly faster, don't know why but I doubt it improves performance as it probably has to do with an underfit model.

I added the base translations for you, you should be able to take it from there.
PS. I do think the quality is sub-par, see also this issue

thanks for the reply mate! its quite interesting, mainly im trying to use this method to pull up a models cohearance for jp > en translation, i found while looking on HF some oasst2 japanese datasets, is it possible to use somthing like this for example to feed into it ?
https://huggingface.co/datasets/kunishou/oasst2-135k-ja
https://huggingface.co/datasets/kunishou/oasst1-89k-ja
i do see some LLM's have previously been trained on the seccond one
thanks for all your efforts though, has been super fun diving into this on consumer hardware

You should be able to use those yes, if you modify the create threads script to read "text_ja" column instead of "text" you should be good to go already by the looks of it.
If not: I tried madlad for translation, seems to work well for Japanese but it needs to be loaded quantized to work on a 16GB card/Colab.

how were you able to use the madlad dataset for it? did you use this ? allenai/MADLAD-400 or am i overcomplicating it ? i did download the new code set just not sure where to point it currently?  thank you :)

It uses google/madlad400-3b-mt, be sure to use --use_madlad and optionally --madlad_quant


store repos in a sub folder based on the first character

closes #651 and #646
splits repos out into groups based on the first letter, we still need to figure out how to migrate existing projects without too much trouble. This could also cause our backups to blow up, I've asked Tim what the fallout might be.

I strongly recommend forbidding - as the first character in project codes; see https://github.com/sillsdev/languageforge-lexbox/issues/651#issuecomment-2016369874 for the reason why. If we can't do that, we will need to special-case project codes that start with - and put them in a folder that isn't named -, which will complicate the logic of this PR.

Just realized I still need to update the command runner script.

So, I did a bit of work on this, because I checked it out to play with it.
I wouldn't be surprised if someone feels the need to clean up my shell command lol.
It seems brace expansion isn't supported in the common light-weight linux images.
I got Python errors when I tried to look at the project page, because the folder listed under [paths] didn't exist.

Also, for the record, I couldn't understand the python code enough to understand why the PR makes the difference 😉
https://repo.mercurial-scm.org/hg/file/tip/mercurial/hgweb/hgwebdir_mod.py#l293
https://repo.mercurial-scm.org/hg/file/tip/mercurial/scmutil.py#l441

Alright I've added a script to go with the one Tim made which will move all our folders, it will leave symlinks in place so we could actually run this before we update any code on our servers. Then once we update the code we can remove all the symlinks. @rmunn would you take a look at the bash script in the hg-deployment.yaml?

@rmunn would you take a look at the bash script in the hg-deployment.yaml?
Looking at it now.

It seems brace expansion isn't supported in the common light-weight linux images.

Because you're running with sh -c. Brace expansion is a bash feature, and not found in the barebones sh. Replace that with bash -c and it should work.

So this is the plan?

Cherry-pick the new init-containers into main
Push that to staging
Verify
Push it to prod
Verify
Push the rest to staging
Verify
Push the rest to prod
Verify

It seems that sh also doesn't support for folder in /repos/[a-z0-9]?*/, so I rewrote it.
While I was testing I rearranged the script a bit per some of @rmunn's thoughts/suggestions.
The current script worked for me on both an existing volume as well as a brand new deployment. Here's an example result:

It seems that sh also doesn't support for folder in /repos/[a-z0-9]?*/, so I rewrote it. While I was testing I rearranged the script a bit per some of @rmunn's thoughts/suggestions.

Huh. The sources I checked said that the [a-z] glob syntax was valid in ash, which is what busybox uses as its shell. But maybe busybox trimmed that part as too rarely used? Anyway, the find command looks correct, I just need to brush up on the subtleties of settings IFS to the empty string before I give the script a :+1:.

FYI to rollback the moving of the projects this will do what we need find . -type l -exec sh -c 'target="$(readlink -f "$0")"; rm "$0"; mv "$target" "$0"' {} \;


Cannot use null safety < Dart 2.10.0

With the latest master, using any null safety syntax does not work with:
analyzer:
  enable-experiment:
    - non-nullable

The analyzer says that the SDK constraint needs to be >=2.10.0 and then pub get needs to be run, however, the latest available Dart version is 2.10.0-0.0.dev.

Hi @creativecreatorormaybenot
With the current master you need the latest SDK version in your pubspec.yaml:
environment:
  sdk: ">=2.10.0-0.0.dev.flutter-24c7666def <3.0.0"

Let me know if this help you.

@pedromassango I did do that, but id does not help unfortunately 😕

@pedromassangocode Did work after invalidating caches 👍🏽

Does this compile to release?

@creativecreatorormaybenot flutter clean or something else for invalidating cache?

@ollydixon It was an IDE problem. Invalidate caches & restart worked for me in AS. There is no real issue here. If you set the SDK constraint correctly, you will not have a problem.

@creativecreatorormaybenot ah I'm using VSC getting the same issue but there's no cache cause it's extremely lightweight hehe.

@creativecreatorormaybenot do guys know how close Dart is to releasing this? (Where can I find out hehe)


Fix API rename issue in shim layer that calls EVP_MAC_CTX_set_params

Master is currently failng due to a compile issue related to this.

Checklist

[ ] documentation is added or updated
[ ] tests are added or updated

There is a config build that is timing out all the time now.. I wonder if this is CLI hang related??

Thanks. Merged to master.


osx linker warnings about chained fixups

ld: warning: -undefined dynamic_lookup may not work with chained fixups
https://github.com/bazelbuild/bazel/issues/16413
In particular see this comment: https://github.com/bazelbuild/bazel/issues/16413#issuecomment-1270603343
Warning seen on macos-12, macos-11 have different unrelated warnings (ld: warning: dylib (/Users/runner/hostedtoolcache/Python/3.10.13/x64/lib/libpython3.10.dylib) was built for newer macOS version (11.7) than being linked (11.0))

Issues seen in simplebuild-dgcode CI.

More relevant info:
https://developer.apple.com/forums/thread/719961
It might indeed be OK to simply ignore this warning.

Seems we should do like everyone else compiling python extensions, and ignore this for the time being. Closing.


Some arguably useful document manipulation features

Hi again,
as announced in #271, we've been developing some features to allow some basic manipulation of existing documents.
I have supplied them with unit tests, end user documentation and in-code explanation for the most tricky part. They do not aim at perfection: I know there is still plenty of room for improvement, feel free to use this as a start if you like to.
Also feel free to ask me anything if some parts are not clear. If you need more to produce more documentation, I will try to.
Thank you!

I see it failed python 2.6. No problem, I can make it 2.6 compatible if this is going to be merged :)

No worries @gtozzi, we can attend to that when we get to working on these. Thanks for contributing these :)


feat(talents & events): Added conditional rendering

Added conditional rendering to Talent and Event pages.

Looks good mate! Congrats on the PR :)


Minor: add test + docs for 2 argument trunc with columns

Which issue does this PR close?
Related to https://github.com/apache/arrow-datafusion/pull/6942 from @Syleechan
Rationale for this change
While reviewing https://github.com/apache/arrow-datafusion/pull/6942 I noticed the two argument version of trunc wasn't tested via sql and it wasn't documented
What changes are included in this PR?
add test + docs for 2 argument trunc with columns
Are these changes tested?
Yes

Are there any user-facing changes?

@Syleechan  would you have time to review this PR?

Thanks @viirya  and @Syleechan  - I have addressed your comments.


beyonce.batchWriteWithTransaction with Updates

Is there a way to do  batch write with updates, puts and deletes ?
I think I red on the doc that conditional updates are not yet possible to ?!
Right now I am using the dynamodb client to make something like this work :
client.transactWrite({
    TransactItems : [
        {
            "Put": {
                "TableName": `${process.env.DATA_TABLE}`,
                "Item": {
                    "pk": {"S": `${BeginsWith.User}-${userId}`},
                    "sk": {"S": `${BeginsWith.Friend}-${firendId}`},
                    "followedUser": {"S": userId},
                    "followingUser": {"S": firendId},
                    "timestamp": {"S": +new Date},
                },
                "ConditionExpression": "attribute_not_exists(sk)",
                "ReturnValuesOnConditionCheckFailure": "ALL_OLD",
            }
        },
        {
            "Update": {
                "TableName": `${process.env.DATA_TABLE}`,
                "Key": {"pk": {"S": `${BeginsWith.User}-${firendId}`}, "sk": {"S": `${BeginsWith.User}-${firendId}`}},
                "UpdateExpression": "SET followers = followers + :i",
                "ExpressionAttributeValues": {":i": {"N": "1"}},
                "ReturnValuesOnConditionCheckFailure": "ALL_OLD",
            }
        },
        {
            "Update": {
                "TableName": `${process.env.DATA_TABLE}`,
                "Key": {"pk": {"S": `${BeginsWith.User}-${userId}`}, "sk": {"S": `${BeginsWith.User}-${userId}`}},
                "UpdateExpression": "SET following = following + :i",
                "ExpressionAttributeValues": {":i": {"N": "1"}},
                "ReturnValuesOnConditionCheckFailure": "ALL_OLD",
            }
        },
    ]
})

Thank you so much again for this library, Beyonce rocks !

Related to the same update use case, partial updates like this does not work, because the lens does not have access to the actual value. Can the generated expression detect that and add a update expression with + :1 ?
                let user = await db.update(UserModel.key({ id: userId }), (user)=>{
                    user.followeesNbr =+1
                })

Hey,

Is there a way to do batch write with updates, puts and deletes ?

Not currently, no. Right now we support puts + deletes via batchWrite, but not (yet) updates.

Related to the same update use case, partial updates like this does not work, because the lens does not have access to the actual value. Can the generated expression detect that and add a update expression with + :1 ?

This isn't currently supported, but should be possible. Dynamo supports an ADD operation in update expressions. So it'd be a matter of teaching the proxy object we use for updates how to generate ADD statements.
Of the top of my head, I'm not sure if JS proxies can differentiate between foo.a = 1 and foo.a += 1, so it might have to look something more like foo.a.add(1).
I'm out of office for the next month, so will be slow to respond, but either of these we'd be happy to accept a PR for. Otherwise, I can probably get to this when I'm back.


Example for converting an array to CSV string

I am having some trouble figuring out how to convert an array of objects to a CSV string. All the examples/documentation involves  file output, while I want just a CSV string as output. How do I do that?
Thanks.

You will have to implement a writable stream object in pass it to csv.toStream().


keep-alive is not regarded if previous request was done without it

I'm aware that request library has been deprecated and does not accept any new fixes. The sole purpose of this ticket is to save time to other users of request lib who might run into the same issue and spend hours/days debugging their code.
The problem is with pooled connections. If you do a first request without keep-alive option then a pool entry is created and all future requests to the same host use the same agent. Unfortunately, the request lib does not distinguish if keep-alive option was are wan't specified when trying to match agent from the pool. So what you get is actually an agent without keep-alive and you can experience flaky behaviour when http connections die without an apparent reason. What's worse is that the connection gets dropped without the client knowing that it isn't active anymore. So it's waiting for data/events that never come. This is exactly what happens with official nodejs client for kubernetes (https://github.com/kubernetes-client/javascript).
While the fix would be relatively easy (change the pool ID to incorporate information about the keepalive value), I will try to work around the problem in the client by using explicit AgentClass with a different name (this new name will become part of pool ID then).
I suspect that the issue reported one year ago could have the same root cause: https://github.com/request/request/issues/3257

@jkryl thank you for your research into this, I think I may be also having the same issue as it seems I'm not getting all watch events when using (https://github.com/kubernetes-client/javascript). I will also look into this as it appears some have tried adding fixes but I still get the same issue even in the latest 0.14.3


[Technical Question] How can I stop the iot edge device from sending MQTT messages up stream ?

How can we stop forwarding our MQTT messages to the IoT Hub ?
We have a scenario that our devices are on a production line, and due to network traffic restrictions and cost implications, we would like to have only local MQTT message routing, and not have it send upstream.
What we found was that when we create a automatic deployment in the IoT Hub portal, the common_route is added by default, but for our local E2E test environment, one can safely just omit this message route.
Is there a possibility to stop this ? I hope my question is understandable, and i'm open to explain in more detail.

@gauss345
Apologies for the late reply - I think this is something you'd have to ask IoT Edge team about, as this repository is just about the Python SDK, but your problem appears to be more about the IoT Edge solution configuration.
I spoke with someone from Edge on your behalf, and they told me that you should do the same thing in your production environment as you have done in your local E2E test environment - omit the route.
They also provided me this document to help explain how it works: https://learn.microsoft.com/en-us/azure/iot-edge/module-composition?view=iotedge-1.4#declare-routes
I am going to close this issue now as it has nothing to do with this Python SDK. If anything else comes up, please direct your query towards the Edge product.


pathlib isn't supported anymore

Looks like pathlib is not actively maintained anymore. They recommend an updated pathlib2 library
https://pypi.python.org/pypi/pathlib2/

Thanks @kain88-de! From this discussion, it looks like what we need is to just use pathlib2 as our import instead of pathlib. Since we want to support < python 3.4 for now still, we'll do this.


Add typings for the isLevelEnabled(string) and isXXXEnabled() functions

Please tell us about your environment:

winston version?

[ ] winston@2
[x] winston@3

Language? TypeScript 3.3

What is the problem?
The type definitions do not exist for the isLevelEnabled(string) and isXXXEnabled() functions on the Logger class added by https://github.com/winstonjs/winston/pull/1352
What do you expect to happen instead?
Update the Logger interface to contain isLevelEnabled(string) and isXXXEnabled()  functions: https://github.com/winstonjs/winston/blob/master/index.d.ts#L84

Happy to approve a PR if you’d like to give it a try, otherwise I will likely fix this at some point :)


Second Derivative Clarification

https://github.com/JenningsYe/newton-practice/blob/c727020d28d1b54bf2524458e965b6f24d176abb/newton.py#L15-L16
Can you explain how you got this second derivative?

I explicitly defined it from the second order central difference, but can definitely call the 1st derivative twice instead.