typo in workflow metro map

Description of the bug
In the metro map diagram, ARCTIC should be ARTIC (no "C")
Command used and terminal output
No response
Relevant files
No response
System information
No response

ARCTIC corrected to ARTIC in metromap


Add Permission classes to Views

Add default authentication to views.
Example for viewset, modelviewset, apiview
from rest_framework import permissions

class SomeViewSet(SomeViewType):
    permission_classes = (permissions.SomePermission,)

Example for function based
@permission_classes(permissions.SomePermission,)
def some_veiw():
    ...

Need to define what the defaults should be (if any) and what the command syntax will be.

Decided this isn't something we need right away. Generators gives you a starting point, not a production ready API.


[#5262] Fix data replication query and filter unused storage resource

Resolves #5262

Pull Request Checklist:

[ ] Positive Test Case Written by Dev

[ ] Automated Testing

[ ] Sufficient User and Developer Documentation

[ ] Passing Jenkins Build

[ ] Peer Code review and approval

Positive Test Case
In order to properly test this locally, one would have to install and run the quota microservice, or manually set quota AVUs. I think it will be best to test on beta

@cbcunc requesting your review specifically on the irods query:
https://github.com/hydroshare/hydroshare/pull/5443/files#diff-42ba409ab49f71821e3fde2cdcb18cddec43cbc17b1eeea1ee2cde95b04c1c16R445-R462
@pkdash requesting your review of the management command:
https://github.com/hydroshare/hydroshare/pull/5443/files#diff-69668d21608defcd1a90ec04154a27a9979e97bc948909fec1c96943b41e5236

@pkdash and @cbcunc
I tested this on beta today and verified that the new filesize query fixes the resources that I listed here:
https://github.com/hydroshare/hydroshare/issues/5262#issuecomment-2039951727

@pkdash I'm requesting a re-review. I added filtering by size in https://github.com/hydroshare/hydroshare/pull/5443/commits/a9709c05db8f0a1fb071695bd17244a12573c18f

@pkdash I ran a new test of the management command to update filesizes on beta with --update --min_quota_use 5 and it took longer than I anticipated.
So I added progress reporting (print current quota number out of total) and csv write in each loop iteration (instead of writing the csv all at once at the end of the command) in case things get interrupted. Here is the diff where I added those:
https://github.com/hydroshare/hydroshare/pull/5443/files/d9d8220d9d60b7d86f66d726580d7d3adef7bd7b..745c8240c537cc4ef94fe38132473fef3fe126ff
I'm wondering if maybe we should track the dateTime of the most recent updating of filesizes at the resource level...so that we can filter out resources that have already been recently updated. Similar to what we do for resource_repair
https://github.com/hydroshare/hydroshare/blob/2.13/hs_core/models.py#L2102-L2104
What do you think of that idea?


[BUG] deploy should not save auto-created folders into retrieve folder

Is there an existing issue for this?

[X] I have searched the existing issues

Current Behavior
when deploying things, folders get auto-created in the deploy-folder - but when those are actually deployed, they also get automatically stored in the retrieve folder.
Expected Behavior
deploy should only save metadata types to retrieve folder that are mentioned in the auto-retrieve list in the config
Steps To Reproduce

Go to '...'
Click on '....'
Run '...'
See error...

Version
7.3.1
Environment

OS:
Node:
npm:

Participation

[X] I am willing to submit a pull request for this issue.

Additional comments
No response

Duplicate of #126


Improve breadcrumbs on Application (for applicants)

Resolves #5368

This was much easier when I remembered we already have the custom_breadcrumbs mechanic 😅


fix(ui): field type being overridden when providing a Cell component

🚀 This is included in version v3.0.0-beta.105


Javascript Odd Even Number Checker

Thanks for the PR. The website looks dope. It has been merged.
I will be glad if you can leave a star on this repository.


Generic treehash

Add generic treehash - generic over hashing algos.
This includes the fastest Sandy Bridge and Skylake hash functions (over long strings) that I have seen yet. These are not rigorously checked for correctness yet, but they are incremental changes to other, simpler, treehashes.

Numbers I get on Skylake processor...
$ ./variablelengthbenchmark
#Reporting the number of bytes per cycle.
#First number is input length in  8-byte words.
0 "&hashVHASH64" "&CLHASH" "&hashCity" "&hashSipHash" "&GHASH64bit" "(&generic_treehash<BoostedZeroCopyGenericBinaryTreehash, Wide<CLNH, 7> >)" "(&generic_treehash<BoostedZeroCopyGenericBinaryTreehash, Wide<NH, 9> >)"
(...)
  64 		 1.88  2.88  1.95  0.43  0.95  2.39  1.94

I found graphing the whole output (ala fig 1(a) from the 2015 paper) interesting. Treehash performance is sawtooth-like, but surpasses CLHASH in speed at somewhere between a few hundred and a few thousand elements.
I think the intra-run variability comes from the "rollup" function. I think the inter-run variability comes from the instruction cache.

I think the intra-run variability comes from the "rollup" function. I think the inter-run variability comes from the instruction cache.

Can you elaborate?

Well, "rollup" behaves very differently when there are a lot of half-full levels in "workspace". In particular, it needs extra time because it calls T::Hash more often.
And when selecting a different set of functions to test, the icache faces different pressures. More (and longer) functions lead to more cache pressure.
I'm not sure that these explain the performance variability. I haven't even measured the inter-run variance.


Fix the return types in the ApiBankingAliases class

This fix the issue #187

Thanks very much @bilel-noomene :-)


feat(h): support jsx syntax

fixes https://github.com/TylorS167/mostly-dom/issues/50

[x] I added new tests for the issue I fixed or the feature I built
[x] I ran npm test for the package I'm modifying

I'm not used to manipulating all those typescript interfaces, so I may have messed up somewhere.
Anyway, with this autocomplete works for native tags, and there's some work to do for it to work on custom components, not sure how yet.

I'm in China for 3 months, I won't be able to help. Do whatever you want
with this PR
Le 6 mars 2018 9:30 AM, "Frederik Krautwald"<EMAIL_ADDRESS>a
écrit :

@Frikki commented on this pull request.
In src/hyperscript/h-jsx.test.tsx
https://github.com/TylorS167/mostly-dom/pull/52#discussion_r172385152:

+import * as assert from 'assert'

+import { hJsx as h } from './h-jsx'
+import { HyperscriptChildren } from '../index'
+
+describe('h jsx support', () => {

it('accepts a function', () => {
function MyComponent() {

 return <div />

}

const vNode =
assert.strictEqual(vNode.tagName, 'div')
})

it('supports destructuration', () => {

cc @TylorS167 https://github.com/tylors167
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/TylorS167/mostly-dom/pull/52#discussion_r172385152,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ABvePppFBBmTA-gcESHWfRLzRLy6AeFfks5tbeacgaJpZM4Q7jP1
.

Hey @Sinewyk I took your work here and merged it in with #63. I really appreciate your work here. I've recently become a TSX convert and will be very glad to use it with mostly-dom going forward


Suggestion: Using terminal friendly column sizes

Just cloned this, noticed that there were some long lines, and thought that it may be helpful for others. :)

Closing this since it's been 2 years without reply from @freebsdgirl. Of course, feel free to reopen if you want the changes. Thanks!


Add support transfer some special tokens of Acala, Karura, Interlay, Bifrost

Overview
Acala, Karura, Interlay, Bifrost now support multi tokens.

Acala: ACA (main token), aUSD, DOT, LDOT...
Karura: kUSD, KSM....
...
I success to create API transfer token in Moonbeam base network, now we need to support some thing like that for above network

Solution

I found this guild to transfer with Acala in substrate way (https://wiki.acala.network/integrate/integration/token-transfer#send-tokens)
Karuara will be the same with Acala
Please research more about transfer token in another networks.

Todo

Create new API for token transfer () in packages/extension-koni-base/src/api/dotsama/transfer.ts

[X] Interlay, Kintsugi, Kintsugi testnet
[x] Acala, Karura

Modify checkTransfer and makeTransfer to impletement new API in same interface with other transfers

[X] Interlay, Kintsugi, Kintsugi testnet
[x] Acala, Karura

[x] Modify function getFreeBalance in 'api/dotsama/balance.ts'
Update UI #143

Bifrost will implement in #234


401 Unauthorized on a cluster with X-Pack enabled

I'm experiencing following issue with this image...
2017-05-03T15:08:36.192683683Z 2017-05-03 15:08:36 +0000 [warn]: temporarily failed to flush the buffer. next_retry=2017-05-03 15:09:06 +0000 error_class="Elasticsearch::Transport::Transport::Errors::Unauthorized" error="[401] " plugin_id="object:2b1bd5d5c250"

I do have the following config in place
---
apiVersion: extensions/v1beta1
kind: DaemonSet
metadata:
  name: fluentd
  namespace: kube-system
  labels:
    k8s-app: fluentd-logging
spec:
  template:
    metadata:
      labels:
        k8s-app: fluentd-logging
    spec:
      containers:
      - name: fluentd
        image: fluent/fluentd-kubernetes-daemonset:v0.12-elasticsearch
        resources:
          requests:
            cpu: 100m
            memory: 200Mi
          limits:
            memory: 200Mi
        env:
        - name: FLUENT_ELASTICSEARCH_HOST
          value: "elasticsearch"
        - name: FLUENT_ELASTICSEARCH_PORT
          value: "9200"
        - name: FLUENT_ELASTICSEARCH_USER
          value: "elastic"
        - name: FLUENT_ELASTICSEARCH_PASSWORD
          value: "changeme"
        volumeMounts:
        - name: fluentconfig
          mountPath: /home/fluent/fluentd/etc/
        - name: varlog
          mountPath: /var/log
        - name: varlibdockercontainers
          mountPath: /var/lib/docker/containers
          readOnly: true
      terminationGracePeriodSeconds: 30
      volumes:
      - name: varlog
        hostPath:
          path: /var/log
      - name: varlibdockercontainers
        hostPath:
          path: /var/lib/docker/containers
      - name: fluentconfig
        configMap:
          name: fluent-config

In order to handle the FLUENT_ELASTICSEARCH_USER and FLUENT_ELASTICSEARCH_PASSWORD I have added the following configmap.
---
apiVersion: v1
kind: ConfigMap
metadata:
  name: fluent-config
  namespace: kube-system
data:
  fluent.conf: |
    @include kubernetes.conf

    <match **>
      type elasticsearch
      log_level info
      include_tag_key true
      host "#{ENV['FLUENT_ELASTICSEARCH_HOST']}"
      port "#{ENV['FLUENT_ELASTICSEARCH_PORT']}"
      user "#{ENV['FLUENT_ELASTICSEARCH_USER']}"
      password "#{ENV['FLUENT_ELASTICSEARCH_PASSWORD']}"
      logstash_format true
      buffer_chunk_limit 2M
      buffer_queue_limit 32
      flush_interval 5s
      max_retry_wait 30
      disable_retry_limit
      num_threads 8
    </match>
  kubernetes.conf: |
    <match fluent.**>
      type null
    </match>

    <source>
      type tail
      path /var/log/containers/*.log
      pos_file /var/log/fluentd-containers.log.pos
      time_format %Y-%m-%dT%H:%M:%S.%NZ
      tag kubernetes.*
      format json
      read_from_head true
    </source>

    ..................
    ................
    .... remainder of kubernetes.conf left for brevity

Is there anyone else experiencing this issue, or knows how to handle the BASIC auth properly?
Is it actually supported by the elasticsearch plugin on this image?

I found the issue. The volumeMount moundPath should not have the tailing slash.
mountPath: /home/fluent/fluentd/etc/ should be mountPath: /home/fluent/fluentd/etc

sorry, not using Elasticsearch as a logging target

Any update on this one?

The following commit should address your problem:
https://github.com/fluent/fluentd-kubernetes-daemonset/commit/64a5ba0bd46b8f86c13559f0338ebd5b734de841
Let me know how it goes.

Fixed.


gem-master HA

Currently, the gem-master is a single point of failure. The services that comprise the gem-master should be highly available and independently scalable. https://github.com/gemini-project/gemini/issues/48 should be considered as a prerequisite to making the gem-master HA. Additionally, HA is being added to the contrib/ansible project. As part of this effort, haproxy and keepalived roles are being added to the project. It should be preferred to use these roles to avoid maintaining separate HA service functionality.

/cc: @stephenrlouie @adamschaub

kubernetes/contrib#761 for kube-master HA.


SVG demo holds CPU at 100% utilization indefinitely

System Information

Windows 10, OS build 19044.1586
Intel Core i7-7500U CPU @ 2.90 GHz
Intel HD Graphics 620 (Integrated), 8 GB VRAM
16 GB RAM, 2133 MHz

Description
Running cargo run --release -p demo -- svg assets/svgs/paris-30k.svg successfully compiles and displays the expected SVG map, however, all 4 logical processors are pegged at 100%, indefinitely, until I terminate the program.

That's working as intended: the SVG demo re-draws everything every frame so you can get an idea of the performance of rendering something as large as the partis-30k map. Animating each element of the map should have similar performance for example.


Parallel deployments

FYI @bratseth @tokle

Nothing to add.

@bratseth PTAL

Realized that we now need to handle jobs that complete out of order (56a639a996a25dc4c77529017c18d11cb7db41c8).

PTAL.


Ouster + External IMU inconsistent rendering and frequent jumps

I have been trying to integrate my Ouster OS1-16 (Gen1) with an XSENS MTi-680G and here's how I set them up:

Ouster OS1 is connected to the Ethernet port of laptop
MTi IMU sensor is connected to a USB port
I have set the Ouster to timestamp in PTP 1588 mode, and issued the command sudo ptpd -i enp3s0 -M in a Terminal to make the Ethernet chip broadcast timings (thanks to @px4Li for the tip here)

Tested and it seems PTP timestamps are working correctly but when I input the bag file data into LIO-SAM I get the following result with jumps and displacements:

I have read the README.md and un-synced timestamps are stated as a cause for these issues but I here's a sample output from the 3 first messages of topics for LiDAR, internal IMU and XSENS IMU in the bag and as you can see the timings are pretty close, are these considered out of sync? Or am I missing something else?
LiDAR (t) timestamp:<PHONE_NUMBER>310819517 - March 1, 2021 4:37:19.310 PM
LiDAR (msg.header) timestamp:<PHONE_NUMBER>254676480 - March 1, 2021 4:37:19.254 PM
LiDAR (t) timestamp:<PHONE_NUMBER>374563536 - March 1, 2021 4:37:19.374 PM
LiDAR (msg.header) timestamp:<PHONE_NUMBER>304678400 - March 1, 2021 4:37:19.304 PM
LiDAR (t) timestamp:<PHONE_NUMBER>423463862 - March 1, 2021 4:37:19.423 PM
LiDAR (msg.header) timestamp:<PHONE_NUMBER>354687744 - March 1, 2021 4:37:19.354 PM

Internal IMU (t) timestamp:<PHONE_NUMBER>282313498 - March 1, 2021 4:37:19.282 PM
Internal IMU (msg.header) timestamp:<PHONE_NUMBER>280500775 - March 1, 2021 4:37:19.280 PM
Internal IMU (t) timestamp:<PHONE_NUMBER>291110001 - March 1, 2021 4:37:19.291 PM
Internal IMU (msg.header) timestamp:<PHONE_NUMBER>290500695 - March 1, 2021 4:37:19.290 PM
Internal IMU (t) timestamp:<PHONE_NUMBER>301079360 - March 1, 2021 4:37:19.301 PM
Internal IMU (msg.header) timestamp:<PHONE_NUMBER>300518531 - March 1, 2021 4:37:19.300 PM

XSENS IMU (t) timestamp:<PHONE_NUMBER>298836081 - March 1, 2021 4:37:19.298 PM
XSENS IMU (msg.header) timestamp:<PHONE_NUMBER>298028272 - March 1, 2021 4:37:19.298 PM
XSENS IMU (t) timestamp:<PHONE_NUMBER>299337757 - March 1, 2021 4:37:19.299 PM
XSENS IMU (msg.header) timestamp:<PHONE_NUMBER>298052150 - March 1, 2021 4:37:19.298 PM
XSENS IMU (t) timestamp:<PHONE_NUMBER>299362726 - March 1, 2021 4:37:19.299 PM
XSENS IMU (msg.header) timestamp:<PHONE_NUMBER>298077029 - March 1, 2021 4:37:19.298 PM

rosbag info of the bag:
version:     2.0
duration:    21.5s
start:       Mar 01 2021 20:07:19.28<PHONE_NUMBER>.28)
end:         Mar 01 2021 20:07:40.78<PHONE_NUMBER>.78)
size:        327.3 MB
messages:    17140
compression: none [431/431 chunks]
types:       sensor_msgs/Imu         [6a62c6daae103f4ff57a132d6f95cec2]
             sensor_msgs/NavSatFix   [2d3a8cd499b9b4a0249fb98fd05cfa48]
             sensor_msgs/PointCloud2 [1158d486dd51d683ce2f1be655c3c181]
             tf2_msgs/TFMessage      [94810edda583a504dfda3829e70d7eec]
topics:      /gnss                     72 msgs @   3.9 Hz : sensor_msgs/NavSatFix
             /imu/data               7243 msgs @ 33.8 kHz : sensor_msgs/Imu
             /os_cloud_node/imu      2151 msgs @ 100.0 Hz : sensor_msgs/Imu
             /os_cloud_node/points    430 msgs @  21.1 Hz : sensor_msgs/PointCloud2
             /tf                     7243 msgs @ 35.5 kHz : tf2_msgs/TFMessage
             /tf_static                 1 msg             : tf2_msgs/TFMessage

Here's LIO-SAM's output:
[ WARN] [1614621691.228977708,<PHONE_NUMBER>.476415178]: Large velocity, reset IMU-preintegration!
[ WARN] [1614621696.328799517,<PHONE_NUMBER>.572775929]: Large velocity, reset IMU-preintegration!
[ WARN] [1614621698.231852286,<PHONE_NUMBER>.479392595]: Large velocity, reset IMU-preintegration!
[ WARN] [1614621698.950174892,<PHONE_NUMBER>.201077672]: Large velocity, reset IMU-preintegration!
[ WARN] [1614621699.709356978,<PHONE_NUMBER>.953501557]: Large velocity, reset IMU-preintegration!
[ WARN] [1614621700.532156937,<PHONE_NUMBER>.786435385]: Large velocity, reset IMU-preintegration!
[ WARN] [1614621701.204714765,<PHONE_NUMBER>.455877594]: Large velocity, reset IMU-preintegration!
[ WARN] [1614621701.948003696,<PHONE_NUMBER>.195174552]: Large velocity, reset IMU-preintegration!
[ WARN] [1614621703.001573379,<PHONE_NUMBER>.247956986]: Large velocity, reset IMU-preintegration!
[ WARN] [1614621704.086892839,<PHONE_NUMBER>.075745433]: Large velocity, reset IMU-preintegration!
[ WARN] [1614621728.522772966,<PHONE_NUMBER>.864422705]: Failed to meet update rate! Took<PHONE_NUMBER>0700000088
[ WARN] [1614621728.524362000,<PHONE_NUMBER>.864422705]: Failed to meet update rate! Took<PHONE_NUMBER>0699999065

And here's my params.yaml:
lio_sam:

  # Topics
  pointCloudTopic: "/os_cloud_node/points"               # Point cloud data
  imuTopic: "/imu/data"                         # IMU data
  odomTopic: "odometry/imu"                   # IMU pre-preintegration odometry, same frequency as IMU
  gpsTopic: "odometry/gpsz"                   # GPS odometry topic from navsat, see module_navsat.launch file

  # Frames
  lidarFrame: "os_lidar"
  baselinkFrame: "base_link"
  odometryFrame: "odom"
  mapFrame: "map"

  # GPS Settings
  useImuHeadingInitialization: false           # if using GPS data, set to "true"
  useGpsElevation: false                      # if GPS elevation is bad, set to "false"
  gpsCovThreshold: 2.0                        # m^2, threshold for using GPS data
  poseCovThreshold: 25.0                      # m^2, threshold for using GPS data

  # Export settings
  savePCD: true                              # https://github.com/TixiaoShan/LIO-SAM/issues/3
  savePCDDirectory: "/Documents/PointClouds"        # in your home folder, starts and ends with "/". Warning: the code deletes "LOAM" folder then recreates it. See "mapOptimization" for implementation

  # Sensor Settings
  sensor: ouster                            # lidar sensor type, either 'velodyne' or 'ouster'
  N_SCAN: 16                                  # number of lidar channel (i.e., 16, 32, 64, 128)
  Horizon_SCAN: 1024                          # lidar horizontal resolution (Velodyne:1800, Ouster:512,1024,2048)
  downsampleRate: 1                           # default: 1. Downsample your data if too many points. i.e., 16 = 64 / 4, 16 = 16 / 1
  lidarMinRange: 1.0                          # default: 1.0, minimum lidar range to be used
  lidarMaxRange: 1000.0                       # default: 1000.0, maximum lidar range to be used

  # IMU Settings
  imuAccNoise: 3.9939570888238808e-03
  imuGyrNoise: 1.5636343949698187e-03
  imuAccBiasN: 6.4356659353532566e-05
  imuGyrBiasN: 3.5640318696367613e-05
  imuGravity: 9.80511
  imuRPYWeight: 0.01

  # Extrinsics (lidar -> IMU)
  extrinsicTrans: [0.0, 0.0, 0.0]

  extrinsicRot: [1, 0, 0,
                 0, 1, 0,
                 0, 0, 1]

  extrinsicRPY: [1, 0, 0,
                 0, 1, 0,
                 0, 0, 1]

  # LOAM feature threshold
  edgeThreshold: 1.0
  surfThreshold: 0.1
  edgeFeatureMinValidNum: 10
  surfFeatureMinValidNum: 100

  # voxel filter paprams
  odometrySurfLeafSize: 0.4                     # default: 0.4 - outdoor, 0.2 - indoor
  mappingCornerLeafSize: 0.2                    # default: 0.2 - outdoor, 0.1 - indoor
  mappingSurfLeafSize: 0.4                      # default: 0.4 - outdoor, 0.2 - indoor

  # robot motion constraint (in case you are using a 2D robot)
  z_tollerance: 1000                            # meters
  rotation_tollerance: 1000                     # radians

  # CPU Params
  numberOfCores: 4                              # number of cores for mapping optimization
  mappingProcessInterval: 0.15                  # seconds, regulate mapping frequency

  # Surrounding map
  surroundingkeyframeAddingDistThreshold: 1.0   # meters, regulate keyframe adding threshold
  surroundingkeyframeAddingAngleThreshold: 0.2  # radians, regulate keyframe adding threshold
  surroundingKeyframeDensity: 2.0               # meters, downsample surrounding keyframe poses
  surroundingKeyframeSearchRadius: 50.0         # meters, within n meters scan-to-map optimization (when loop closure disabled)

  # Loop closure
  loopClosureEnableFlag: true
  loopClosureFrequency: 1.0                     # Hz, regulate loop closure constraint add frequency
  surroundingKeyframeSize: 50                   # submap size (when loop closure enabled)
  historyKeyframeSearchRadius: 15.0             # meters, key frame that is within n meters from current pose will be considerd for loop closure
  historyKeyframeSearchTimeDiff: 30.0           # seconds, key frame that is n seconds older will be considered for loop closure
  historyKeyframeSearchNum: 25                  # number of hostory key frames will be fused into a submap for loop closure
  historyKeyframeFitnessScore: 0.3              # icp threshold, the smaller the better alignment

  # Visualization
  globalMapVisualizationSearchRadius: 1000.0    # meters, global map visualization radius
  globalMapVisualizationPoseDensity: 10.0       # meters, global map visualization keyframe density
  globalMapVisualizationLeafSize: 1.0           # meters, global map visualization cloud density

# Navsat (convert GPS coordinates to Cartesian)
navsat:
  frequency: 50
  wait_for_datum: false
  delay: 0.0
  magnetic_declination_radians: 0
  yaw_offset: 0
  zero_altitude: true
  broadcast_utm_transform: false
  broadcast_utm_transform_as_parent_frame: false
  publish_filtered_gps: false

# EKF for Navsat
ekf_gps:
  publish_tf: false
  map_frame: map
  odom_frame: odom
  base_link_frame: base_link
  world_frame: odom

  frequency: 50
  two_d_mode: false
  sensor_timeout: 0.01
  # -------------------------------------
  # External IMU:
  # -------------------------------------
  imu0: imu_correct
  # make sure the input is aligned with ROS REP105. "imu_correct" is manually transformed by myself. EKF can also transform the data using tf between your imu and base_link
  imu0_config: [false, false, false,
                true,  true,  true,
                false, false, false,
                false, false, true,
                true,  true,  true]
  imu0_differential: false
  imu0_queue_size: 50
  imu0_remove_gravitational_acceleration: true
  # -------------------------------------
  # Odometry (From Navsat):
  # -------------------------------------
  odom0: odometry/gps
  odom0_config: [true,  true,  true,
                 false, false, false,
                 false, false, false,
                 false, false, false,
                 false, false, false]
  odom0_differential: false
  odom0_queue_size: 10

  #                            x     y     z     r     p     y   x_dot  y_dot  z_dot  r_dot p_dot y_dot x_ddot y_ddot z_ddot
  process_noise_covariance: [  1.0,  0,    0,    0,    0,    0,    0,     0,     0,     0,    0,    0,    0,    0,      0,
                               0,    1.0,  0,    0,    0,    0,    0,     0,     0,     0,    0,    0,    0,    0,      0,
                               0,    0,    10.0, 0,    0,    0,    0,     0,     0,     0,    0,    0,    0,    0,      0,
                               0,    0,    0,    0.03, 0,    0,    0,     0,     0,     0,    0,    0,    0,    0,      0,
                               0,    0,    0,    0,    0.03, 0,    0,     0,     0,     0,    0,    0,    0,    0,      0,
                               0,    0,    0,    0,    0,    0.1,  0,     0,     0,     0,    0,    0,    0,    0,      0,
                               0,    0,    0,    0,    0,    0,    0.25,  0,     0,     0,    0,    0,    0,    0,      0,
                               0,    0,    0,    0,    0,    0,    0,     0.25,  0,     0,    0,    0,    0,    0,      0,
                               0,    0,    0,    0,    0,    0,    0,     0,     0.04,  0,    0,    0,    0,    0,      0,
                               0,    0,    0,    0,    0,    0,    0,     0,     0,     0.01, 0,    0,    0,    0,      0,
                               0,    0,    0,    0,    0,    0,    0,     0,     0,     0,    0.01, 0,    0,    0,      0,
                               0,    0,    0,    0,    0,    0,    0,     0,     0,     0,    0,    0.5,  0,    0,      0,
                               0,    0,    0,    0,    0,    0,    0,     0,     0,     0,    0,    0,    0.01, 0,      0,
                               0,    0,    0,    0,    0,    0,    0,     0,     0,     0,    0,    0,    0,    0.01,   0,
                               0,    0,    0,    0,    0,    0,    0,     0,     0,     0,    0,    0,    0,    0,      0.015]

A couple of things to note:

I have upgraded the firmware on both sensors, Ouster recently released version 2.0 (changelog available here), not sure if this is causing incompatibility issues with the current version of LIO-SAM.
One strange thing in the rosbag info output of the ros bag is that /imu/data frequency is stated as 33 kHz while I set it to 400 MHz inside the device settings manager!

If anyone could give me some pointers, I'd be grateful.

I also tried the same setup except this time setting the Ouster to timestamp using its internal clock (TIME_FROM_INTERNAL_OSC) but when I play the bag file I get an error in RViZ stating Message removed because it is too old (frame=[os_sensor], stamp=[7410.940159360]).
Here are the first 3 messages from this bag:
LiDAR (t) timestamp:<PHONE_NUMBER>904756332 - March 1, 2021 4:40:44.904 PM
LiDAR (msg.header) timestamp:<PHONE_NUMBER>010 	    - August 8, 2204 7:32:22.010 PM
LiDAR (t) timestamp:<PHONE_NUMBER>954839447 - March 1, 2021 4:40:44.954 PM
LiDAR (msg.header) timestamp:<PHONE_NUMBER>220 	    - August 9, 2204 9:25:09.220 AM
LiDAR (t) timestamp:<PHONE_NUMBER>999718884 - March 1, 2021 4:40:44.999 PM
LiDAR (msg.header) timestamp:<PHONE_NUMBER>890	    - August 9, 2204 11:18:03.890 PM

Internal IMU (t) timestamp:<PHONE_NUMBER>868188755 - March 1, 2021 4:40:44.868 PM
Internal IMU (msg.header) timestamp:<PHONE_NUMBER>520       - August 9, 2204 3:42:58.520 AM
Internal IMU (t) timestamp:<PHONE_NUMBER>878109171 - March 1, 2021 4:40:44.878 PM
Internal IMU (msg.header) timestamp:<PHONE_NUMBER>580       - August 9, 2204 6:29:38.580 AM
Internal IMU (t) timestamp:<PHONE_NUMBER>887993798 - March 1, 2021 4:40:44.887 PM
Internal IMU (msg.header) timestamp:<PHONE_NUMBER>810       - August 9, 2204 9:16:18.810 AM

XSENS IMU (t) timestamp:<PHONE_NUMBER>866481557 - March 1, 2021 4:40:44.866 PM
XSENS IMU (msg.header) timestamp:<PHONE_NUMBER>865712708 - March 1, 2021 4:40:44.865 PM
XSENS IMU (t) timestamp:<PHONE_NUMBER>866509771 - March 1, 2021 4:40:44.866 PM
XSENS IMU (msg.header) timestamp:<PHONE_NUMBER>865768865 - March 1, 2021 4:40:44.865 PM
XSENS IMU (t) timestamp:<PHONE_NUMBER>866521682 - March 1, 2021 4:40:44.866 PM
XSENS IMU (msg.header) timestamp:<PHONE_NUMBER>865800048 - March 1, 2021 4:40:44.865 PM

set imu 100hz or 200hz, Adjust the parameters

Hі, Hummblebee! І have the same problem with os1-16 sensor.

@jiawenquan I have not tried with 100Hz but here's the result with IMU 20 Hz to make it on par with the LiDAR frequency:

Will try 100Hz later and report back.

Hі, Hummblebee! І have the same problem with os1-16 sensor.

Well it seems like Ouster issues with LIO-SAM are common, I'm still researching, if I find a fix I'll let you know ASAP.

Hі, Hummblebee! І have the same problem with os1-16 sensor.

You may set too many unnecessary imu options, causing communication blockage

I have been trying to integrate my Ouster OS1-16 (Gen1) with an XSENS MTi-680G and here's how I set them up:
* Ouster OS1 is connected to the Ethernet port of laptop

* MTi IMU sensor is connected to a USB port

* I have set the Ouster to timestamp in PTP 1588 mode, and issued the command `sudo ptpd -i enp3s0 -M` in a Terminal to make the Ethernet chip broadcast timings (thanks to @px4Li for the tip [here](https://github.com/TixiaoShan/LIO-SAM/issues/207#issuecomment-787873217))

Tested and it seems PTP timestamps are working correctly but when I input the bag file data into LIO-SAM I get the following result with jumps and displacements:

I have read the README.md and un-synced timestamps are stated as a cause for these issues but I here's a sample output from the 3 first messages of topics for LiDAR, internal IMU and XSENS IMU in the bag and as you can see the timings are pretty close, are these considered out of sync? Or am I missing something else?
LiDAR (t) timestamp:<PHONE_NUMBER>310819517 - March 1, 2021 4:37:19.310 PM
LiDAR (msg.header) timestamp:<PHONE_NUMBER>254676480 - March 1, 2021 4:37:19.254 PM
LiDAR (t) timestamp:<PHONE_NUMBER>374563536 - March 1, 2021 4:37:19.374 PM
LiDAR (msg.header) timestamp:<PHONE_NUMBER>304678400 - March 1, 2021 4:37:19.304 PM
LiDAR (t) timestamp:<PHONE_NUMBER>423463862 - March 1, 2021 4:37:19.423 PM
LiDAR (msg.header) timestamp:<PHONE_NUMBER>354687744 - March 1, 2021 4:37:19.354 PM

Internal IMU (t) timestamp:<PHONE_NUMBER>282313498 - March 1, 2021 4:37:19.282 PM
Internal IMU (msg.header) timestamp:<PHONE_NUMBER>280500775 - March 1, 2021 4:37:19.280 PM
Internal IMU (t) timestamp:<PHONE_NUMBER>291110001 - March 1, 2021 4:37:19.291 PM
Internal IMU (msg.header) timestamp:<PHONE_NUMBER>290500695 - March 1, 2021 4:37:19.290 PM
Internal IMU (t) timestamp:<PHONE_NUMBER>301079360 - March 1, 2021 4:37:19.301 PM
Internal IMU (msg.header) timestamp:<PHONE_NUMBER>300518531 - March 1, 2021 4:37:19.300 PM

XSENS IMU (t) timestamp:<PHONE_NUMBER>298836081 - March 1, 2021 4:37:19.298 PM
XSENS IMU (msg.header) timestamp:<PHONE_NUMBER>298028272 - March 1, 2021 4:37:19.298 PM
XSENS IMU (t) timestamp:<PHONE_NUMBER>299337757 - March 1, 2021 4:37:19.299 PM
XSENS IMU (msg.header) timestamp:<PHONE_NUMBER>298052150 - March 1, 2021 4:37:19.298 PM
XSENS IMU (t) timestamp:<PHONE_NUMBER>299362726 - March 1, 2021 4:37:19.299 PM
XSENS IMU (msg.header) timestamp:<PHONE_NUMBER>298077029 - March 1, 2021 4:37:19.298 PM

rosbag info of the bag:
version:     2.0
duration:    21.5s
start:       Mar 01 2021 20:07:19.28<PHONE_NUMBER>.28)
end:         Mar 01 2021 20:07:40.78<PHONE_NUMBER>.78)
size:        327.3 MB
messages:    17140
compression: none [431/431 chunks]
types:       sensor_msgs/Imu         [6a62c6daae103f4ff57a132d6f95cec2]
             sensor_msgs/NavSatFix   [2d3a8cd499b9b4a0249fb98fd05cfa48]
             sensor_msgs/PointCloud2 [1158d486dd51d683ce2f1be655c3c181]
             tf2_msgs/TFMessage      [94810edda583a504dfda3829e70d7eec]
topics:      /gnss                     72 msgs @   3.9 Hz : sensor_msgs/NavSatFix
             /imu/data               7243 msgs @ 33.8 kHz : sensor_msgs/Imu
             /os_cloud_node/imu      2151 msgs @ 100.0 Hz : sensor_msgs/Imu
             /os_cloud_node/points    430 msgs @  21.1 Hz : sensor_msgs/PointCloud2
             /tf                     7243 msgs @ 35.5 kHz : tf2_msgs/TFMessage
             /tf_static                 1 msg             : tf2_msgs/TFMessage

Here's LIO-SAM's output:
[ WARN] [1614621691.228977708,<PHONE_NUMBER>.476415178]: Large velocity, reset IMU-preintegration!
[ WARN] [1614621696.328799517,<PHONE_NUMBER>.572775929]: Large velocity, reset IMU-preintegration!
[ WARN] [1614621698.231852286,<PHONE_NUMBER>.479392595]: Large velocity, reset IMU-preintegration!
[ WARN] [1614621698.950174892,<PHONE_NUMBER>.201077672]: Large velocity, reset IMU-preintegration!
[ WARN] [1614621699.709356978,<PHONE_NUMBER>.953501557]: Large velocity, reset IMU-preintegration!
[ WARN] [1614621700.532156937,<PHONE_NUMBER>.786435385]: Large velocity, reset IMU-preintegration!
[ WARN] [1614621701.204714765,<PHONE_NUMBER>.455877594]: Large velocity, reset IMU-preintegration!
[ WARN] [1614621701.948003696,<PHONE_NUMBER>.195174552]: Large velocity, reset IMU-preintegration!
[ WARN] [1614621703.001573379,<PHONE_NUMBER>.247956986]: Large velocity, reset IMU-preintegration!
[ WARN] [1614621704.086892839,<PHONE_NUMBER>.075745433]: Large velocity, reset IMU-preintegration!
[ WARN] [1614621728.522772966,<PHONE_NUMBER>.864422705]: Failed to meet update rate! Took<PHONE_NUMBER>0700000088
[ WARN] [1614621728.524362000,<PHONE_NUMBER>.864422705]: Failed to meet update rate! Took<PHONE_NUMBER>0699999065

And here's my params.yaml:
lio_sam:

  # Topics
  pointCloudTopic: "/os_cloud_node/points"               # Point cloud data
  imuTopic: "/imu/data"                         # IMU data
  odomTopic: "odometry/imu"                   # IMU pre-preintegration odometry, same frequency as IMU
  gpsTopic: "odometry/gpsz"                   # GPS odometry topic from navsat, see module_navsat.launch file

  # Frames
  lidarFrame: "os_lidar"
  baselinkFrame: "base_link"
  odometryFrame: "odom"
  mapFrame: "map"

  # GPS Settings
  useImuHeadingInitialization: false           # if using GPS data, set to "true"
  useGpsElevation: false                      # if GPS elevation is bad, set to "false"
  gpsCovThreshold: 2.0                        # m^2, threshold for using GPS data
  poseCovThreshold: 25.0                      # m^2, threshold for using GPS data

  # Export settings
  savePCD: true                              # https://github.com/TixiaoShan/LIO-SAM/issues/3
  savePCDDirectory: "/Documents/PointClouds"        # in your home folder, starts and ends with "/". Warning: the code deletes "LOAM" folder then recreates it. See "mapOptimization" for implementation

  # Sensor Settings
  sensor: ouster                            # lidar sensor type, either 'velodyne' or 'ouster'
  N_SCAN: 16                                  # number of lidar channel (i.e., 16, 32, 64, 128)
  Horizon_SCAN: 1024                          # lidar horizontal resolution (Velodyne:1800, Ouster:512,1024,2048)
  downsampleRate: 1                           # default: 1. Downsample your data if too many points. i.e., 16 = 64 / 4, 16 = 16 / 1
  lidarMinRange: 1.0                          # default: 1.0, minimum lidar range to be used
  lidarMaxRange: 1000.0                       # default: 1000.0, maximum lidar range to be used

  # IMU Settings
  imuAccNoise: 3.9939570888238808e-03
  imuGyrNoise: 1.5636343949698187e-03
  imuAccBiasN: 6.4356659353532566e-05
  imuGyrBiasN: 3.5640318696367613e-05
  imuGravity: 9.80511
  imuRPYWeight: 0.01

  # Extrinsics (lidar -> IMU)
  extrinsicTrans: [0.0, 0.0, 0.0]

  extrinsicRot: [1, 0, 0,
                 0, 1, 0,
                 0, 0, 1]

  extrinsicRPY: [1, 0, 0,
                 0, 1, 0,
                 0, 0, 1]

  # LOAM feature threshold
  edgeThreshold: 1.0
  surfThreshold: 0.1
  edgeFeatureMinValidNum: 10
  surfFeatureMinValidNum: 100

  # voxel filter paprams
  odometrySurfLeafSize: 0.4                     # default: 0.4 - outdoor, 0.2 - indoor
  mappingCornerLeafSize: 0.2                    # default: 0.2 - outdoor, 0.1 - indoor
  mappingSurfLeafSize: 0.4                      # default: 0.4 - outdoor, 0.2 - indoor

  # robot motion constraint (in case you are using a 2D robot)
  z_tollerance: 1000                            # meters
  rotation_tollerance: 1000                     # radians

  # CPU Params
  numberOfCores: 4                              # number of cores for mapping optimization
  mappingProcessInterval: 0.15                  # seconds, regulate mapping frequency

  # Surrounding map
  surroundingkeyframeAddingDistThreshold: 1.0   # meters, regulate keyframe adding threshold
  surroundingkeyframeAddingAngleThreshold: 0.2  # radians, regulate keyframe adding threshold
  surroundingKeyframeDensity: 2.0               # meters, downsample surrounding keyframe poses
  surroundingKeyframeSearchRadius: 50.0         # meters, within n meters scan-to-map optimization (when loop closure disabled)

  # Loop closure
  loopClosureEnableFlag: true
  loopClosureFrequency: 1.0                     # Hz, regulate loop closure constraint add frequency
  surroundingKeyframeSize: 50                   # submap size (when loop closure enabled)
  historyKeyframeSearchRadius: 15.0             # meters, key frame that is within n meters from current pose will be considerd for loop closure
  historyKeyframeSearchTimeDiff: 30.0           # seconds, key frame that is n seconds older will be considered for loop closure
  historyKeyframeSearchNum: 25                  # number of hostory key frames will be fused into a submap for loop closure
  historyKeyframeFitnessScore: 0.3              # icp threshold, the smaller the better alignment

  # Visualization
  globalMapVisualizationSearchRadius: 1000.0    # meters, global map visualization radius
  globalMapVisualizationPoseDensity: 10.0       # meters, global map visualization keyframe density
  globalMapVisualizationLeafSize: 1.0           # meters, global map visualization cloud density

# Navsat (convert GPS coordinates to Cartesian)
navsat:
  frequency: 50
  wait_for_datum: false
  delay: 0.0
  magnetic_declination_radians: 0
  yaw_offset: 0
  zero_altitude: true
  broadcast_utm_transform: false
  broadcast_utm_transform_as_parent_frame: false
  publish_filtered_gps: false

# EKF for Navsat
ekf_gps:
  publish_tf: false
  map_frame: map
  odom_frame: odom
  base_link_frame: base_link
  world_frame: odom

  frequency: 50
  two_d_mode: false
  sensor_timeout: 0.01
  # -------------------------------------
  # External IMU:
  # -------------------------------------
  imu0: imu_correct
  # make sure the input is aligned with ROS REP105. "imu_correct" is manually transformed by myself. EKF can also transform the data using tf between your imu and base_link
  imu0_config: [false, false, false,
                true,  true,  true,
                false, false, false,
                false, false, true,
                true,  true,  true]
  imu0_differential: false
  imu0_queue_size: 50
  imu0_remove_gravitational_acceleration: true
  # -------------------------------------
  # Odometry (From Navsat):
  # -------------------------------------
  odom0: odometry/gps
  odom0_config: [true,  true,  true,
                 false, false, false,
                 false, false, false,
                 false, false, false,
                 false, false, false]
  odom0_differential: false
  odom0_queue_size: 10

  #                            x     y     z     r     p     y   x_dot  y_dot  z_dot  r_dot p_dot y_dot x_ddot y_ddot z_ddot
  process_noise_covariance: [  1.0,  0,    0,    0,    0,    0,    0,     0,     0,     0,    0,    0,    0,    0,      0,
                               0,    1.0,  0,    0,    0,    0,    0,     0,     0,     0,    0,    0,    0,    0,      0,
                               0,    0,    10.0, 0,    0,    0,    0,     0,     0,     0,    0,    0,    0,    0,      0,
                               0,    0,    0,    0.03, 0,    0,    0,     0,     0,     0,    0,    0,    0,    0,      0,
                               0,    0,    0,    0,    0.03, 0,    0,     0,     0,     0,    0,    0,    0,    0,      0,
                               0,    0,    0,    0,    0,    0.1,  0,     0,     0,     0,    0,    0,    0,    0,      0,
                               0,    0,    0,    0,    0,    0,    0.25,  0,     0,     0,    0,    0,    0,    0,      0,
                               0,    0,    0,    0,    0,    0,    0,     0.25,  0,     0,    0,    0,    0,    0,      0,
                               0,    0,    0,    0,    0,    0,    0,     0,     0.04,  0,    0,    0,    0,    0,      0,
                               0,    0,    0,    0,    0,    0,    0,     0,     0,     0.01, 0,    0,    0,    0,      0,
                               0,    0,    0,    0,    0,    0,    0,     0,     0,     0,    0.01, 0,    0,    0,      0,
                               0,    0,    0,    0,    0,    0,    0,     0,     0,     0,    0,    0.5,  0,    0,      0,
                               0,    0,    0,    0,    0,    0,    0,     0,     0,     0,    0,    0,    0.01, 0,      0,
                               0,    0,    0,    0,    0,    0,    0,     0,     0,     0,    0,    0,    0,    0.01,   0,
                               0,    0,    0,    0,    0,    0,    0,     0,     0,     0,    0,    0,    0,    0,      0.015]

A couple of things to note:
* I have upgraded the firmware on both sensors, Ouster recently released version 2.0 ([changelog available here](https://data.ouster.io/downloads/changelog/changelog-v2p0.pdf)), not sure if this is causing incompatibility issues with the current version of LIO-SAM.

* One strange thing in the `rosbag info` output of the ros bag is that `/imu/data` frequency is stated as 33 kHz while I set it to 400 MHz inside the device settings manager!

If anyone could give me some pointers, I'd be grateful.

I noticed that the rotational speed of your OS1-16 is 20 Hz. however, Mr. Tixiao Shan's description says that the file imageProjection.cpp "" deskewPoint () "function utilizes this relative time to obtain the transformation of this point relative to the beginning of the scan. When the lidar rotates at 10Hz, the timestamp of a point should vary between 0 and 0.1 seconds. If you are using other lidar sensors, you may need to change the name of this time channel and make sure that it is the relative time in a scan. "
Perhaps this is preventing the correct timing of the point cloud and imu synchronization? Or maybe you should try  OS-1 16 in 10 Hz mode?

@hummblebee   Hі, Hummblebee!  I am testing the lio-sam system with subt datase but I got a continuous warning for "Large velocity, reset IMU-preintegration!". And the trajectory result is very bad. Did you test lio-sam with subt dataset？

@MaxikKot Following your suggestions, I tried recording with LiDAR in 10Hz and different IMU frequencies including 100Hz but the issue still persists!
I got this error message from Google Cartographer ROS maybe it gives someone a clue:
Check failed: msg->linear_acceleration_covariance[0] != -1 (-1 vs. -1) Your IMU data claims to not contain linear acceleration measurements by setting linear_acceleration_covariance[0] to -1. Cartographer requires this data to work. See http://docs.ros.org/api/sensor_msgs/html/msg/Imu.html.

@hummblebee Hі, Hummblebee! I am testing the lio-sam system with subt datase but I got a continuous warning for "Large velocity, reset IMU-preintegration!". And the trajectory result is very bad. Did you test lio-sam with subt dataset？

Hi @bugdeveloper0 no I have not, I did try with a Velodyne dataset I downloaded from the internet and also a sample small dataset from Ouster's website and they both produced good results.

I found this excerpt from XSENS knowledge base on integrating MTi sensors with Velodyne LiDARS:

In contrast to the MTi-G-710, the MTi-680G does not yet offer a “true” 1 PPS output that comes straight from the internal GNSS receiver. Instead, by using the Interval Transition Measurement synchronization feature, the MTi-680G can be configured to generate its own 1 PPS signal that is synchronized with the 1 PPS signal of the internal receiver. This pulse will be synchronized with the internal GNSS 1 PPS pulse in terms of frequency, but not in terms of phase. This means that the 1 PPS output of the MTi-680G does not appear at the exact start of each UTC second. The timing of the pulse depends on the moment you power up the MTi-680G.

The MTi-680G does provide sub-second data in its NMEA messages, however some lidar brands do not copy the full UTC time information from the $GPGGA or $GPRMC packets: They often assume that the 1 PPS signal and its corresponding data packet coincide with the start of a UTC second, and therefore the sub-seconds field is assumed to be zero. This can cause a data timing error of up to 1 second.

Further guidelines for the MTi-680G will be added as soon as it is able to output a "true" GNSS 1 PPS signal.

Need expert opinion on this, do you believe this is the reason I'm getting bad results?

Values in matrices  extrinsicRot: and extrinsicRPY correspond to the location of the imu relative to the ouster?

I found this excerpt from XSENS knowledge base on integrating MTi sensors with Velodyne LiDARS:

In contrast to the MTi-G-710, the MTi-680G does not yet offer a “true” 1 PPS output that comes straight from the internal GNSS receiver. Instead, by using the Interval Transition Measurement synchronization feature, the MTi-680G can be configured to generate its own 1 PPS signal that is synchronized with the 1 PPS signal of the internal receiver. This pulse will be synchronized with the internal GNSS 1 PPS pulse in terms of frequency, but not in terms of phase. This means that the 1 PPS output of the MTi-680G does not appear at the exact start of each UTC second. The timing of the pulse depends on the moment you power up the MTi-680G.

The MTi-680G does provide sub-second data in its NMEA messages, however some lidar brands do not copy the full UTC time information from the $GPGGA or $GPRMC packets: They often assume that the 1 PPS signal and its corresponding data packet coincide with the start of a UTC second, and therefore the sub-seconds field is assumed to be zero. This can cause a data timing error of up to 1 second.

Further guidelines for the MTi-680G will be added as soon as it is able to output a "true" GNSS 1 PPS signal.

Need expert opinion on this, do you believe this is the reason I'm getting bad results?

As I understand from your previous messages, you did not connect MTi-G-710 to ouster via pps, but synchronized via ptp? if so, what is described cannot be the cause of bad results.

@hummblebee Hі, Hummblebee! I am testing the lio-sam system with subt datase but I got a continuous warning for "Large velocity, reset IMU-preintegration!". And the trajectory result is very bad. Did you test lio-sam with subt dataset？

Hi @bugdeveloper0 no I have not, I did try with a Velodyne dataset I downloaded from the internet and also a sample small dataset from Ouster's website and they both produced good results.

Hi, @hummblebee
Do you think there is mapping problem because we use ptpd?
According to the issue https://github.com/TixiaoShan/LIO-SAM/issues/94, it says that the timestamp should be synchronized precisely.
Shouldn't we use GPS or hardware PTP to synchronize timestamp?

Values in matrices extrinsicRot: and extrinsicRPY correspond to the location of the imu relative to the ouster? Does the Horizon_SCAN value correspond to the horizontal resolution of the scanner at 10 Hz?

I'm not sure about the extrinsic matrices, I went ahead with identity matrices just to rule out any misalignments but I highly doubt that identity matrices cause jumps! I set the Ouster's LiDAR mode to 2048x10 so Horizon_SCAN is set to 2048 currently (but was 1024 when I first created this issue).

@hummblebee Hі, Hummblebee! I am testing the lio-sam system with subt datase but I got a continuous warning for "Large velocity, reset IMU-preintegration!". And the trajectory result is very bad. Did you test lio-sam with subt dataset？

Hi @bugdeveloper0 no I have not, I did try with a Velodyne dataset I downloaded from the internet and also a sample small dataset from Ouster's website and they both produced good results.

Hi, @hummblebee
Do you think there is mapping problem because we use ptpd?
According to the issue #94, it says that the timestamp should be synchronized precisely.
Shouldn't we use GPS or hardware PTP to synchronize timestamp?

@JJuhyeon-Kim It may be, I can't think of anything else. If someone could upload a bag file of an Ouster and an external IMU synced properly I could analyze the timings and maybe come up with a solution.

I managed to sync the timestamps and here's what I got:

The jumps are gone! YAY! But the projected data is kind of vibrating in-place! I did see #120 but that one is not distorted like my case, as if the vibration does not register a new cloud over the previous one, but this is the case in mine!

Try to record your bag outdoors rather than indoors. And  set the white noise parameters for your INS in params.yaml. imuAccNoise etc.

Hi, @hummblebee !
First of all, thank you for always answering my questions kindly even though you must be very busy.
I think I solved the problem.
I saw in the ouster driver that ros timestamp is not accurate.
So I didn't consider the issue.
However, using a ros timestamp was effective for me. (I recommend you to check issue#https://github.com/TixiaoShan/LIO-SAM/issues/94, the commend of YushengWHU)
My imu sensor is the same as the author's, and imu sensor's timestamp is also ros timestamp.
When I used ros timestamp, the rostopic delay was much smaller than when I used ptpd.

If you want to use ros timestamp, then check this issue https://github.com/ouster-lidar/ouster_example/issues/74.
The figure below is what I obtained.

Thank you for your kindness.
I hope your research goes well!

Hi @JJuhyeon-Kim
Thanks for sharing your experience, and sorry for the late reply! I'm glad you got it working. Unfortunately I abandoned the project as it was consuming too much time and getting nowhere, but I'll leave this issue open as long as possible, if it is any help for others.
Good luck to you all.


Bug in argument parsing in zsh when setopt EXTENDED_GLOB

Rust Version
rustc 1.26.1 (827013a31 2018-05-25)
Affected Version of clap
2.31
Behaviour and Steps
If you run this gist in zsh like this:
setop EXTENDED_GLOB && example --value ^ *

it should output:
Input: [{files in the directory from which you launch it}]
Value: "^"

Instead it outputs:
Input: [{files the directory from which you launch it}]
Value: {first file in the directory from which you launch it}

The caret is treated as a globbing pattern even though it's in an "option" argument, not "input" argument.
It seems that clap doesn't distinguish between values of "input" (that is, files, which might need globbing) and values of "options" (which never need globbing).

Note that the same happens in python's argparse. I'm not sure if it's considered a bug or not.

On second though, this is not clap's bug.


Author style clustering?

Might be interesting to add author-style clustering based on:
https://huggingface.co/datasets/MiMe-MeMo/Corpus-v1.1

That's actually a great idea we probs have a dataset for that in-house don't we?


Uncaught TypeError: fileType.toLowerCase is not a function

[Enter steps to reproduce:]

just open atom
...but after modifying the config file in probably an inadequate fashion. can't open it anymore...

Atom: 1.20.0 x64
Electron: 1.6.9
OS: Mac OS X 10.12.6
Thrown From: Atom Core
Stack Trace
Uncaught TypeError: fileType.toLowerCase is not a function
At /Applications/Atom.app/Contents/Resources/app/src/grammar-registry.js:90

TypeError: fileType.toLowerCase is not a function
    at GrammarRegistry.module.exports.GrammarRegistry.getGrammarPathScore (/Applications/Atom.app/Contents/Resources/app/src/grammar-registry.js:90:45)
    at GrammarRegistry.module.exports.GrammarRegistry.getGrammarScore (/Applications/Atom.app/Contents/Resources/app/src/grammar-registry.js:64:26)
    at /Applications/Atom.app/Contents/Resources/app/src/text-editor-registry.js:321:52
    at Set.forEach (native)
    at TextEditorRegistry.grammarAddedOrUpdated (/Applications/Atom.app/Contents/Resources/app/src/text-editor-registry.js:307:47)
    at Function.module.exports.Emitter.simpleDispatch (/Applications/Atom.app/Contents/Resources/app/node_modules/event-kit/lib/emitter.js:25:20)
    at Emitter.module.exports.Emitter.emit (/Applications/Atom.app/Contents/Resources/app/node_modules/event-kit/lib/emitter.js:141:34)
    at GrammarRegistry.module.exports.GrammarRegistry.addGrammar (/Applications/Atom.app/Contents/Resources/app/node_modules/first-mate/lib/grammar-registry.js:76:26)
    at Grammar.module.exports.Grammar.activate (/Applications/Atom.app/Contents/Resources/app/node_modules/first-mate/lib/grammar.js:254:54)
    at /Applications/Atom.app/Contents/Resources/app/src/package.js:686:31
    at /Applications/Atom.app/Contents/Resources/app/node_modules/first-mate/lib/grammar-registry.js:121:61
    at parseContents (/Applications/Atom.app/Contents/Resources/app/node_modules/season/lib/cson.js:123:13)
    at /Applications/Atom.app/Contents/Resources/app/node_modules/season/lib/cson.js:212:26
    at tryToString (fs.js:426:3)
    at FSReqWrap.readFileAfterClose [as oncomplete] (fs.js:413:12)

Commands
     -0:22.8.0 grammar-selector:show (atom-text-editor.editor.is-focused)
     -0:19.8.0 core:confirm (input.hidden-input)

Non-Core Packages
autocomplete-python 1.10.3
compare-files 0.8.1
emmet 2.4.3
file-types 0.5.5
git-clone 0.2.0
git-plus 7.9.3
highlight-selected 0.13.1
python-tools 0.6.9
rhino-python 0.8.4
script 3.15.0

...but after modifying the config file in probably an inadequate fashion. can't open it anymore...

What modification did you make?  Also, can you share your config file so we can take a look?

i wanted to add another entry to customFileTypes and just messed up the brackets. i had a bit a hard time finding it to reverse the change. all is back to normal again! but with the error in it, atom didn't let me edit it anymore and did no syntax-highlighting on any of the files anymore.

Glad you have things resolved :+1:

but with the error in it, atom didn't let me edit it anymore and did no syntax-highlighting on any of the files anymore.


#2780 Support either prop defined as findings or Findings in AWS Security Hub Parser

NOTE: This closes #2780.
This template is for your information. Please clear everything when submitting your pull request.
Note: DefectDojo is now on Python3.6 and Django 2.2.x. Please submit your pull requests to the 'dev' branch as the 'legacy-python2.7' branch is only for bug fixes. Any new features submitted to the legacy branch will be ignored and closed.
When submitting a pull request, please make sure you have completed the following checklist:

[x] Give a meaningful name to your PR, as it may end up being used in the release notes.
[x] Your code is flake8 compliant.
[x] Your code is python 3.6 compliant (specific python >3.6 syntax is currently not accepted).
[x] If this is a new feature and not a bug fix, you've included the proper documentation in the ReadTheDocs documentation folder. https://github.com/DefectDojo/Documentation/tree/master/docs or provide feature documentation in the PR.
[x] Model changes must include the necessary migrations in the dojo/db_migrations folder.
[x] Add applicable tests to the unit tests.
[x] Add the proper label to categorize your PR.

Moderators: Labels currently accepted for PRs:

Import Scans (for new scanners/importers)
enhancement
performance
feature
bugfix
maintenance (a.k.a chores)
dependencies
New Migration (when the PR introduces a DB migration)
settings_changes (when the PR introduces changes or new settings in settings.dist.py)

Contributors: Git Tips
Rebase on dev branch
If the dev branch has changed since you started working on it, please rebase your work after the current dev.
On your working branch mybranch:
git rebase dev mybranch

In case of conflict:
 git mergetool
 git rebase --continue

When everything's fine on your local branch, force push to your myOrigin remote:
git push myOrigin --force-with-lease

To cancel everything:
git rebase --abort

Squashing commits
git rebase -i origin/dev

Replace pick by fixup on the commits you want squashed out
Replace pick by reword on the first commit if you want to change the commit message
Save the file and quit your editor

Force push to your myOrigin remote:
git push myOrigin --force-with-lease

Not sure people will want a unit test to check the switch, but if that is necessary, please let me know and I will do so, thanks!

Yeah I don't think the unit test should change here. It would be really clean to write it as so tree.get('Findings', tree.get('findings', None)) since it removes the need for any new if cases.

Yeah I don't think the unit test should change here. It would be really clean to write it as so tree.get('Findings', tree.get('findings', None)) since it removes the need for any new if cases.

Let it be so, I will update PR shortly.


Implement Quine in Python

Hi, I can do it!

Sure @mmdbalkhi , Quine program are those programs which print itself i.e. its own source code is the output. And yes, this repository welcomes such programs which explains/demonstrates the lesser known features of any programming language ..

Shall I assign you for this issue? @mmdbalkhi

If you want, yes, why not!

Go ahead @mmdbalkhi

That was not Quine Program @mmdbalkhi as I mentioned "Quine program are those programs which print itself i.e. its own source code is the output."
I am merging your code but this issue remains open as the program doesn't fulfill the requirement for this issue.

That was not Quine Program @mmdbalkhi as I mentioned "Quine program are those programs which print itself i.e. its own source code is the output." I am merging your code but this issue remains open as the program doesn't fulfill the requirement for this issue.

I'm sorry I was wrong. Because I asked you above about lesser known capabilities. I'm trying to find something really weird.

No worries :)

Above PR is a self printing program but can't be called Quine. Issue remains open.


KIS has disappeared from CKAN

Hello,
Recently KIS has stopped showing in CKAN. I didn't change any ckan related file before it happen. Though, there were two events that could trigger the issue:

Curseforge which holds the release archive went down couple of times in the last week.
Github repositry owner for KIS mod has changed from KospY to ihsoft. This could the the reason but another mod KAS has exactly same setup and owner change didn't affect it.

I did a check with netkan.exe for the metadata and it worked fine:
netkan.exe  --verbose KIS.netkan

48 [1] INFO CKAN.NetKAN.Program (null) - Transforming KIS.netkan
69 [1] INFO CKAN.NetKAN.Program (null) - Using CKAN cache at k:/steam/SteamApps/common/Kerbal Space Program/CKAN/downloads
100 [1] INFO CKAN.NetKAN.Program (null) - Finished reading input
101 [1] INFO CKAN.NetKAN.Validators.HasIdentifierValidator (null) - Validating that metadata contains an identifier
101 [1] INFO CKAN.NetKAN.Program (null) - Input successfully passed pre-validation
107 [1] INFO CKAN.NetKAN.Transformers.MetaNetkanTransformer (null) - Executing MetaNetkan transformation with #/netkan/https://raw.githubusercontent.com/ihsoft/
KIS/master/KIS.netkan
458 [1] INFO CKAN.NetKAN.Transformers.AvcTransformer (null) - Executing Interal AVC transformation with
612 [1] INFO CKAN.NetKAN.Transformers.AvcTransformer (null) - Found internal AVC version file
612 [1] INFO CKAN.NetKAN.Transformers.AvcTransformer (null) - Found remote AVC version file at https://raw.githubusercontent.com/KospY/KIS/master/KIS.version
646 [1] INFO CKAN.NetKAN.Transformers.AvcTransformer (null) - Remote AVC version file describes same version as local AVC version file, using it preferrentially
.
651 [1] INFO CKAN.NetKAN.Transformers.DownloadAttributeTransformer (null) - Executing Download Attribute transformation with
697 [1] INFO CKAN.NetKAN.Transformers.GeneratedByTransformer (null) - Executing Generated By transformation with
698 [1] INFO CKAN.NetKAN.Transformers.StripNetkanMetadataTransformer (null) - Executing Strip Netkan Metadata transformation with
702 [1] INFO CKAN.NetKAN.Transformers.PropertySortTransformer (null) - Executing property sort transformation
704 [1] INFO CKAN.NetKAN.Program (null) - Finished transformation
709 [1] INFO CKAN.NetKAN.Program (null) - Output successfully passed post-validation
711 [1] INFO CKAN.NetKAN.Program (null) - Transformation written to .\KIS-<IP_ADDRESS>.ckan

As of now I've updated all files obn my side to use new onwer folder on Github. Also, I've submitted changes for .netkan files (https://github.com/KSP-CKAN/NetKAN/pull/5026 and https://github.com/KSP-CKAN/NetKAN/pull/5025).

Probably just a dependendy not being cleared for ksp 1.2.2.
In that case, for some reason the dependent mod does not shot even show up under the "all" tab.
Yep, looking at the netkan file and ckan, CommunityCategoryKit is not yet available for ksp 1.2.2. As soon as that is set to be compatible, KIS will show up again.

Thanks for the investigation. Fixed KSP version for CCK, let's see if it helps. Too bad dependent mods get hidden completely, it confuses players a lot.

It got resolved after I've updated the dependent version.


Tool: JavaFx Exception when launching the tool

When launching the tool to regenerate code, Speedment 3.1.17 casts the following Exception:
aug. 29, 2019 4:53:49 EM com.sun.javafx.css.StyleManager loadStylesheetUnPrivileged INFO: Could not load @font-face font [jar:file:/Users/juliagustafsson/.m2/repository/com/speedment/enterprise/enterprise-tool/3.1.16/enterprise-tool-3.1.16.jar!/fonts/SourceCodePro-Regular.otf] aug. 29, 2019 4:53:49 EM javafx.scene.CssStyleHelper calculateValue WARNING: Caught 'java.lang.ClassCastException: class java.lang.String cannot be cast to class javafx.scene.paint.Paint (java.lang.String is in module java.base of loader 'bootstrap'; javafx.scene.paint.Paint is in unnamed module of loader org.codehaus.plexus.classworlds.realm.ClassRealm @38ed139b)' while converting value for '-fx-background-color' from rule '*.menu-bar' in stylesheet jar:file:/Users/juliagustafsson/.m2/repository/com/speedment/enterprise/enterprise-tool/3.1.16/enterprise-tool-3.1.16.jar!/css/enterprise.css aug. 29, 2019 4:53:49 EM javafx.scene.CssStyleHelper calculateValue WARNING: Caught 'java.lang.ClassCastException: class java.lang.String cannot be cast to class javafx.scene.paint.Paint (java.lang.String is in module java.base of loader 'bootstrap'; javafx.scene.paint.Paint is in unnamed module of loader org.codehaus.plexus.classworlds.realm.ClassRealm @38ed139b)' while converting value for '-fx-background-color' from rule '*.menu-bar *.menu' in stylesheet jar:file:/Users/juliagustafsson/.m2/repository/com/speedment/enterprise/enterprise-tool/3.1.16/enterprise-tool-3.1.16.jar!/css/enterprise.css aug. 29, 2019 4:53:50 EM javafx.scene.CssStyleHelper calculateValue WARNING: Caught 'java.lang.ClassCastException: class java.lang.String cannot be cast to class javafx.scene.paint.Paint (java.lang.String is in module java.base of loader 'bootstrap'; javafx.scene.paint.Paint is in unnamed module of loader org.codehaus.plexus.classworlds.realm.ClassRealm @38ed139b)' while converting value for '-fx-background-color' from rule '*.menu-bar *.menu' in stylesheet jar:file:/Users/juliagustafsson/.m2/repository/com/speedment/enterprise/enterprise-tool/3.1.16/enterprise-tool-3.1.16.jar!/css/enterprise.css

Is there a specific way to recreate this? I can't get it to reproduce the exception locally.

I ran into this error immediately upon launching Speedment. I'll try to run it again and see if it is still an issue.

I realize now that this was probably resolved with release 3.1.17. I must have ran on 3.1.16 unintentionally.


[Windows] Improper quoting of parameters

A common method of counting the number of lines in a file in Windows batch is to use the command find /c /v "" some-file. I tried this using ChildProcess.build, since I need a Windows equivalent to wc -l for compatibility. Here were the results:
irb(main):001:0> require 'childprocess'
=> true
irb(main):002:0> process = ChildProcess.build(*%w[find /c /v "" some-file])
=> #<ChildProcess::Windows::Process:0x00000002bd75e8 @args=["find", "/c", "/v", "\"\"", "some-file"], @started=false, @exit_code=nil, @io=nil, @cwd=nil, @detach=false, @duplex=false, @leader=false, @environment={}>
irb(main):003:0> process.io.inherit!
=> #<IO:<STDERR>>
irb(main):004:0> process.start
=> #<ChildProcess::Windows::Process:0x00000002bd75e8 @args=["find", "/c", "/v", "\"\"", "some-file"], @started=true, @exit_code=nil, @io=#<ChildProcess::Windows::IO:0x00000002bedaf0 @stdout=#<IO:<STDOUT>>, @stderr=#<IO:<STDERR>>>, @cwd=nil, @detach=false, @duplex=false, @leader=false, @environment={}, @pid=192, @handle=#<ChildProcess::Windows::Handle:0x00000002348bc0 @pid=192, @pointer=#<FFI::Pointer address=0x0000000000009c>, @closed=false>>
irb(main):005:0> FIND: Parameter format not correct

The problem is that the "" parameter is being quoted as '""' by ChildProcess::Windows::ProcessBuilder#quote_if_necessary. Single-quotes aren't treated as string delimiters in batch, so this doesn't work.
Why was this quoting introduced in the first place? I imagine it must break plenty of other use cases as well.

You're not running a batch script, you're sending arguments to the find executable directly through CreateProcess. The quoting behaviour was introduced to ensure the spec in this PR passes.
If you want shell behaviour, see my suggestion in #85, which should solve your issue.


Enable CFI

Vanadium, which also runs on arm64, enables is_cfi and use_cfi_cast. Perhaps those options should be enabled for MacOS too for improved security.

https://www.chromium.org/developers/testing/control-flow-integrity/ it's only for Linux and Android

The page seems to imply it only works on Linux on x86, yet it works on Android (arm64). It might work on MacOS too in that case.

I believe this is a discussion to be made in https://github.com/ungoogled-software/ungoogled-chromium as my goal is to make macOS build stay the same with the other OS ones. /cc @networkException
And, as that page doesn't explicitly mention that macOS is supported, I tend to not enable it here, unless we know that official Chromium is built with it as one of the project philosophy says:

ungoogled-chromium retains the default Chromium experience as closely as possible


Remove some redundant text from chapter 11

Thanks for finding this. I have a slightly different fix for this, moving the longer explanation of stroke down after the rect stuff.


Riot Valorant doesn't get cached

Describe the issue you are having
When downloading Valorant through Riot Games Launcher, it doesn't get cached. Steam Games, for example Rocket League are working.
Describe your setup?

DNS Server: pfSense Firewall with <IP_ADDRESS> as upstream DNS
I am using lancache-dns with the pfSense Firewall DNS IP as Upstream DNS.
The IP of the lancache-dns is set as primary DNS on the Windows 11 Client. No secondary DNS Server is set.
Are you running sniproxy
no
DNS Configuration
USE_GENERIC_CACHE=true
LANCACHE_IP=<IP_ADDRESS>
DNS_BIND_IP=<IP_ADDRESS>
UPSTREAM_DNS=<IP_ADDRESS>
CACHE_ROOT=/opt/dlcache
CACHE_DISK_SIZE=500g
CACHE_INDEX_SIZE=500m
CACHE_MAX_AGE=3650d
TZ=Europe/Berlin

Other stuff
When checking iftop on the NIC that is used for caching, i can see while downloading Valorant, that the Traffic is going through the cache. But it doesn't get cached.
I have seen that there already was an issue for this #164 but that didn't help.

Updating this to add that this is still an issue. We see Valorant patching exclusively over HTTPS at the following hostnames:

valorant.secure.dyn.riotcdn.net
bacon.secure.dyn.riotcdn.net

These hostnames are poisoned to point to our cache according to the following file: https://github.com/uklans/cache-domains/blob/31b2ba1e0a7c419327cb97f589b508d78b9aecbf/riot.txt#L5

Maybe @v3n can help?

@v3n anything you can assist with?

Hey folks, I'm no longer at Riot, but have reached out to my old colleagues.

Oops, sorry this slipped off our radar for so long, getting some folks to look at this in the next few weeks.

So for some context, the fetching of the release manifest does require SSL for https://valorant.secure.dyn.riotcdn.net, and that's unavoidable. However, after that initial step, patching should fall back to http://valorant.dyn.riotcdn.net as long as that IP resolves to a non-public IP range such as your 10.. I'm not that familiar with this repo, but would it be possible to set it up such that requests at *.secure.dyn.riotcdn.net skip the lan cache?
If that doesn't resolves the issue I can also dig further from our side

The cache domains entry for lancache dns has had *.dyn.riotcdn.net redirect in it since jul 31st 2019. During that time valorant hasn't been falling back, lol has been working. I haven't looked at it recently but this issue was raised again here in sep 23. I expect its still not working as intended.

Yep looks like we have a suspect, testing out some fixes locally, will let ya folks know when we have a timeline for when the fix will hit live!

Hi everyone,
I'm on the Riot patcher team and worked on the initial HTTP fallback feature for LAN parties some 5 years ago.
Thank you for letting us know about this issue, and sorry for taking so long to address it. We have fixed the problem and you should see caching working again with the next version of Riot Client in early April.
If you encounter issues with caching again, please feel free to ping @xelaadryth or myself and we'll do our best to look into it.

Much appreciated. Its always well received when we are able to tell our LAN goers the developers of their favorite games are trying to help us out.

Hi all,
I just wanted to update this thread and let you know that Riot Client v83 is now live and contains the fix for this issue. You should be able to cache Valorant downloads now.
Please let us know if you are still encountering issues.
Thanks!


Minor cataloger improvements for the CLI

Add ALL (synonym for all), ALL:IMAGE, and ALL:DIRECTORY catalog selectors (somewhat in response to #1776)
Add some more INFO and DEBUG log messages about which catalogers were selected.
Add a sanity check: if 0 catalogers are selected, emit an error.

Also make the Syft CLI fail faster if 0 catalogers are selected

See the individual commits and their corresponding commit messages for details.

Hi @jsquyres thank you for this PR -- I'd like to note that there is another upcoming change (which is taking a while to get to completion) that offers some similar, but possibly better options for enabling certain sets of catalogers: https://github.com/anchore/syft/pull/1383 -- would this work for your use case?

@kzantow Interesting.  I'm not deep enough into the zen of Syft to fully grok how #1383 will result in what you're saying, but you seemed to make reference to the same capability in https://github.com/anchore/syft/issues/1776#issuecomment-1555011570.
Our use case is that we have entirely scanned images up to this point, but now we need to scan a filesystem.  We were a little surprised when scanning an image and a corresponding equivalent filesystem turned up different results (which led to my question in #1776).  Reading the docs and digging in to the source code led to better understanding and resulted in this PR.  To be clear: by examining the source of Syft v0.79.0 and using the --catalogs CLI option, we were able to get the results to be the same.  We can certainly keep doing that, but it's a little tedious to have to keep examining the source code or closely examine all the output from Syft to see exactly which catalogers were used in a given run.

I see that #1383 has had no public code activity in ~6 months.  Is there an estimated timeframe when it will become available in a release?
#1383 is a big PR, and I'm still a relative newbie to the Syft code base.  It doesn't look like there are CLI updates corresponding to the API code changes in that PR.  If that is correct, is there any other ongoing work to include CLI corresponding updates?

@kzantow Ping.

@jsquyres though #1383 is stale, it's quite important to the syft 1.0 work, which I think will be landing within the next 3-ish months. The feature isn't completed in that PR yet, but the API will be accounting for this. There is a follow up task in the PR description to consider https://github.com/anchore/syft/issues/1039 and https://github.com/anchore/syft/issues/1731 . #1383 has implemented a tag-based cataloger selection approach in the API, the next step is to wire it up to the --cataloger CLI option.

@wagoodman @kzantow Using a tag-based cataloger selection sounds fine to me.  If I remove the ALL* stuff from this PR, would the other improvements be helpful?  (e.g., the docs changes, logging messages, and fail fast)

Thanks very much for the contribution @jsquyres. It's really great that you've been improving Syft, but the unfortunate timing of these PRs just so happened to overlap with a couple other analogous but different changes. I don't think there's a lot from this PR that we will be able to use, but we really appreciate the contribution and would encourage you to reach out if there are other things you'd like to work on!


Wrong exit code, when no file was created

I am running avif-cli like this: avif --input='img/*.{jpg,jpeg,png}' --quality 64 --verbose. This works fine, when at least one image will be converted. When all images already have been converted and --overwrite has not been set, avif will return with a status of 1 which will be interpreted as failure.
The culprit seems to be this line, which returns undefined in this case.
https://github.com/lovell/avif-cli/blob/main/lib/convert.js#L33
Changing it for example to return true would fix this issue.

Thanks for reporting, a PR with this fix would be most welcome, if you're able.

Thanks for reporting, a PR with this fix would be most welcome, if you're able.

Sure thing: https://github.com/lovell/avif-cli/pull/17

v0.5.1 now available with your fix, thank you.


chore: release 2.3.0

:robot: I have created a release *beep* *boop*
2.3.0 (2019-07-26)
Bug Fixes

deps: update dependency @octokit/graphql to v3 (#208) (200f710)
deps: update dependency @octokit/request to v5 (#209) (6a79479)

Features

adding support for google-auth-library-java (#203) (f72c930)

This PR was generated with Release Please.

Release is at https://github.com/googleapis/release-please/releases/tag/v2.3.0

The release build has started, the log can be viewed here. :sunflower:

The release build failed! Please investigate!


Export data tier application in Azure not possible (DNN database)

Current result
Try to make backup of the DNN database created as Azure SQL database.

In SQL Server Management Studio click on Export data tier application (generate backup, BACPAK file).

Expected result
BACPAK generation can be done successfully.
Error log
TITLE: Microsoft SQL Server Management Studio
One or more unsupported elements were found in the schema used as part of a data package.
Error SQL46010: Procedure: [dbo].[AddTab] contains a statement that is not supported on Microsoft Azure SQL Database v12. The specific error is: Incorrect syntax near Url.
Error SQL46010: Procedure: [dbo].[UpdateTab] contains a statement that is not supported on Microsoft Azure SQL Database v12. The specific error is: Incorrect syntax near Url.
Error SQL46010: Procedure: [dbo].[DeleteUrlTracking] contains a statement that is not supported on Microsoft Azure SQL Database v12. The specific error is: Incorrect syntax near Url.
Error SQL46010: Procedure: [dbo].[DeleteUrl] contains a statement that is not supported on Microsoft Azure SQL Database v12. The specific error is: Incorrect syntax near Url.
Error SQL46010: Procedure: [dbo].[AddSiteLog] contains a statement that is not supported on Microsoft Azure SQL Database v12. The specific error is: Incorrect syntax near Url.
Error SQL46010: Procedure: [dbo].[GetUrlTracking] contains a statement that is not supported on Microsoft Azure SQL Database v12. The specific error is: Incorrect syntax near Url.
Error SQL46010: Procedure: [dbo].[GetUrls] contains a statement that is not supported on Microsoft Azure SQL Database v12. The specific error is: Incorrect syntax near Url.
Error SQL46010: Procedure: [dbo].[GetUrl] contains a statement that is not supported on Microsoft Azure SQL Database v12. The specific error is: Incorrect syntax near Url.
Error SQL46010: Procedure: [dbo].[SaveTabUrl] contains a statement that is not supported on Microsoft Azure SQL Database v12. The specific error is: Incorrect syntax near Url.
Error SQL46010: Procedure: [dbo].[UpdateUrlTrackingStats] contains a statement that is not supported on Microsoft Azure SQL Database v12. The specific error is: Incorrect syntax near Url.
Error SQL46010: Procedure: [dbo].[AddUrlTracking] contains a statement that is not supported on Microsoft Azure SQL Database v12. The specific error is: Incorrect syntax near Url.
Error SQL46010: Procedure: [dbo].[UpdateUrlTracking] contains a statement that is not supported on Microsoft Azure SQL Database v12. The specific error is: Incorrect syntax near Url.
Error SQL46010: Procedure: [dbo].[AddUrl] contains a statement that is not supported on Microsoft Azure SQL Database v12. The specific error is: Incorrect syntax near Url.
Error SQL46010: Procedure: [dbo].[GetTabUrls] contains a statement that is not supported on Microsoft Azure SQL Database v12. The specific error is: Incorrect syntax near Url.
Error SQL46010: Procedure: [dbo].[UpdateServer] contains a statement that is not supported on Microsoft Azure SQL Database v12. The specific error is: Incorrect syntax near URL.
Error SQL46010: Procedure: [dbo].[AddPackage] contains a statement that is not supported on Microsoft Azure SQL Database v12. The specific error is: Incorrect syntax near Url.
Error SQL46010: Procedure: [dbo].[dnnLinks_UpdateLink] contains a statement that is not supported on Microsoft Azure SQL Database v12. The specific error is: Incorrect syntax near Url.
Error SQL46010: Procedure: [dbo].[dnnLinks_GetLinks] contains a statement that is not supported on Microsoft Azure SQL Database v12. The specific error is: Incorrect syntax near URL.
Error SQL46010: Procedure: [dbo].[UpdatePackage] contains a statement that is not supported on Microsoft Azure SQL Database v12. The specific error is: Incorrect syntax near Url.
Error SQL46010: Procedure: [dbo].[dnnLinks_GetLink] contains a statement that is not supported on Microsoft Azure SQL Database v12. The specific error is: Incorrect syntax near URL.
Error SQL46010: Procedure: [dbo].[dnnLinks_AddLink] contains a statement that is not supported on Microsoft Azure SQL Database v12. The specific error is: Incorrect syntax near Url.
Error SQL46010: View: [dbo].[vw_Tabs] contains a statement that is not supported on Microsoft Azure SQL Database v12. The specific error is: Incorrect syntax near Url.
(Microsoft.SqlServer.Dac)
Additional context
The issues can be fixed in the stored procedures and views by putting "url" reference column into brackets => [url]
Affected version
9.3.2

Was this an upgraded installation?  We have hundreds of DNN Installations that are on Azure DB's successfully.

Hi, I have verified the issue by running a local 9.3.2 vanilla installation on a local server. The problem is caused when using the SSMS 18.2 version.
WORKAROUND: use SSMS 17.9.1 to export the bacpac
Anyway, the final solution should be to change those stored procedures to be fully compatible with the new SSMS versions.

What needs to be done here, is this something to fix in Dnn or a bug with SSMS? If something to fix in Dnn, do we want to make that happen in 10 ?

I just ran into this issue trying to create a BACPAC of a 9.3.2 site on Azure.  Thankfully a coworker still had the old version of SSMS installed, so we were able to backup the site.  Initial searches for an SSMS v17.9.1 installer were unsuccessful.

@valadas I believe it is the name "Url" for a column that isn't properly escaped

Ok, so assigning this as a bug, but I do not have a testbed for this here.

I have an Azure SQL backpac (DNN9.3.2) that we may be forced to move to a VM. Is this issue likely to affect that in some way?

@mitchelsellers I have upgraded from 7.03.02 to 9.04.04 version of DNN and am trying to use the Data Migration Assistant. I have the same issues. Is there an issue with upgrading versions of DNN and  moving to Azure Managed Database?

I'll be working on this later this week

I've upgraded from 7.3? to 9.4 and had this same problem. I tried to change url to  [url] where shown but that broke the site, specifically tabs and css. I'm not sure where to go from here.

Right now the workaround would be to use the older version of SSMS.  Its actually a bug in SSMS not DNN as it is "acceptable" to have queries with this structure now.

FYI, you can download the compatible version of SSMS here: https://docs.microsoft.com/en-us/sql/ssms/release-notes-ssms?view=sql-server-2017#download-ssms-1791
I ran into this issue again and had to go find the installer for the old version to make a backup.

Sooo should we close this issue because it is not Dnn related @mitchelsellers @noahbast

@valadas No, I'm going to try to put a fix for the core in for 9.5.0.
Its a simple fix for us to Play "nice" and we might as well as it really is the true SQL Standard

Yes I got it to work using the old version of SSMS, just as an fyi
Microsoft Data Migration assistant throws the same errors as the new SSMS
On Fri, Jan 10, 2020 at 8:10 AM Mitchel Sellers<EMAIL_ADDRESS>wrote:

@valadas https://github.com/valadas No, I'm going to try to put a fix
for the core in for 9.5.0.
Its a simple fix for us to Play "nice" and we might as well as it really
is the true SQL Standard
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/dnnsoftware/Dnn.Platform/issues/2924?email_source=notifications&email_token=AF6JUXKZMVGQESBQDUVJ42LQ5B6VJA5CNFSM4II3TME2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEIUAY4Q#issuecomment-573049970,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AF6JUXLZTTYG5NTUVXMHLZDQ5B6VJANCNFSM4II3TMEQ
.

@mitchelsellers cool, triaged accordingly


Jcb/callstack optimization

This increases the size of string literals we can parse by ~4x.

I'd rather have a complete solution before merging though, no? As we discussed, we still have many stack issues that we need to consider.

Fixed with a different approach.


Adding purview_reader to sysadmin required/ relevant?

In the first code cell for permission setting the new role purview_reader is added to the sysadmin role. This does not seem to be required for the rest of the process, so is this relevant?

Document Details
⚠ Do not edit this section. It is required for learn.microsoft.com ➟ GitHub issue linking.

ID: e94efbb9-d1ea-c297-f89d-e076848ee8a1
Version Independent ID: 20172377-bf9d-a4a4-0193-4af98e4e460a
Content: Connect to and manage Snowflake - Microsoft Purview
Content Source: articles/purview/register-scan-snowflake.md
Service: purview
Sub-service: purview-data-map
GitHub Login: @linda33wj
Microsoft Alias: jingwang

@jniemer
Thanks for your feedback! We will investigate and update as appropriate.

@jniemer - Are there any other questions I can answer for you?

Hi @whhender
we are happy for your response. Thank you for comments on this issue.
@jniemer
We are going to close this thread as resolved but if there are any further questions regarding the documentation, please tag me in your reply and we will be happy to continue the conversation.


test(cart): add resolved cart guard integration test

PR Checklist
Please check if your PR fulfills the following requirements:

[x] The commit message follows our guidelines: https://github.com/graycoreio/daffodil/blob/master/CONTRIBUTING.md#commit
[x] Tests for the changes have been added (for bug fixes / features)
[ ] Docs have been added / updated (for bug fixes / features)

PR Type
What kind of change does this PR introduce?

[ ] Bugfix
[ ] Feature
[ ] Code style update (formatting, local variables)
[ ] Refactoring (no functional changes, no api changes)
[ ] Build related changes
[ ] CI related changes
[ ] Documentation content changes
[x] Other... Please describe: test

What is the current behavior?

Fixes: #1311
Does this PR introduce a breaking change?
[ ] Yes
[x] No

Other information
Trying to repro a bug but could not. Since I wrote the integration test anyway, might as well merge it. It serves as a good code example.

Otherwise, PR LGTM, thanks for the test.


Add operators for rlc

Please could the following operators be added to the XML when users select a random local clock model?
<operators id="operators" optimizationSchedule="default">
......
	<scaleOperator scaleFactor="0.75" weight="15">
		<parameter idref="localClock.relativeRates"/>
	</scaleOperator>
	<bitFlipOperator weight="15">
			<parameter idref="localClock.changes"/>
	</bitFlipOperator>
.....
</operators>

Could you attach an xml file that has this operator in - the entire thing? I need to make sure it is referenced correctly throughout the xml.

Don't worry - I have an example in the 4 xml files you sent.

They ought to be there automatically, I made it so that all operators and priors are automatically in the logs.

Can you check with the new version that it is as you think it should be then let me know?


Issue with Android power saving features

Hi,
I'm using last plugin version and I'm facing lots of problem, gettings position updates, in Android 7+ when power saving features are enabled.
I mean that PositionChanged event not fire when power saving features are enabled and I have configured it to ask for position updates in background implementing an Android service.
Is possible to ask to user/system to be whitelisted or to ignore this setting for a specific app? Or there is any parameter in the plugin setting to solve this?
Thank you

Did you find anything about?


Warning: port uses /usr/local/var instead of /var

The above warning is printed by the FreeBSD ports framework.
/var should be used instead, regardless of PREFIX.

Since I now use cmake and GNUInstallDirs exclusively, this means that according to this message there is a bug in cmake. I'm using the variable CMAKE_INSTALL_FULL_LOCALSTATEDIR and that variable should contain the absolute path to the var directory to use.
So I think you should file this as a bug in cmake and/or GNUInstallDirs, if they fix this variable, my code will end up in the right place.
BTW, I thought /usr/local/var was a symlink to something in /var. But maybe that used to be the case long ago. Or I simply do not remember that correctly. It's been a long time since I last used FreeBSD as my main OS.

Ok, thanks.
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=267748


Add models from the DARPA SubT challenge

Fixes #281.

ping @gchenfc

Sorry I must have missed this the first time around - I think the target branch is wrong?  Or if you're intentionally trying to keep a separate branch, maybe you can create a separate PR first to merge master (or just perform locally) before this PR so it's easier to read?

Oh nevermind, but how come the "files changed" diffs are all messed up on github?

Yeah that's because I merged in master and Github doesn't seem to like that. It should resolve itself once we merge in robot-models.

I guess I can remove the inner subT folder.

This is fine, but perhaps in the future we should consider also reorganizing the urdfs / sdfs into robot-oriented directories (e.g. subt / spider / a1 / cablerobot / etc) instead of sdf / urdf / subt. Since loading syntax doesn't change depending on the file type, and it doesn't really make sense to have folders for file types and a folder that's categorically named that contains sdfs.

The calling syntax is different. And if we have a URDF of a robot, why would we need an SDF and vice-versa? This is why I felt having a robot-centric naming convention was not helpful since the directories would just have the one file and unnecessarily add more typing to load the file aka why should a user type cablerobot/cablerobot.urdf when they can simply do kUrdfPath/cablerobot.urdf?

Also just realized, why are most of these files except cerberus... in a subt/subT subfolder?

Yeah I don't know what I was thinking (or not thinking). Updated this in the last commit, thanks!


Enhancements Request

Thanks for creating this!
I'm going to use it in our church to overlay lyrics on our broadcast mix, and as I'm testing a couple things come to mind that it would be great to see added:
-Add the option for fade times between slides, instead of straight cuts.
-Make screen output size configurable, so users can force a particular resolution.
-Allow for other text alignments like vertically centred, or horizontally left.
-Some way to have more control on the ProPresenter end to decide where the line break should be on certain slides. 2 spaces might work at the end of a line, as ProPresenter will just ignore them, but if it passes them over the network is another question.
-Ability to auto-capitalize the start of every line.
Thanks again for creating this, I'd been hunting for months for something to pass out ProPresenter lyrics over the network instead of SDI cables. This does it nicely!

All features implemented


Collections made empty appear populated upon page refresh

Deleting all collection items then refreshing the page results in the collection items appearing once again in the CMS but not github

Maybe you haven’t received the latest version of the CMS. Well, the CMS itself is also cached, and unfortunately it’s unavoidable.

The CMS script is cached by the CDN for 10 minutes, so sometimes an old version is served just after a new version is published.

It seems to be happening on /<EMAIL_ADDRESS>

Hmm, so it should be including the fix for #237. Deleting a single entry and multiple entries are the same process internally, and not sure why the problem still occurs. Will check back tomorrow.

Added a workaround to v0.46.2. Not 100% sure it fixes the problem, but my test worked fine.


Verilog code generation example

The paper talks about FPGA and ASIC synthesis from Darkroom programs, but I failed to find anything relevant in the repository :(

Yes, the FPGA/ASIC code generator wasn't released along with the Darkroom language frontend.  Maybe @jbrunhaver can comment.
You might want to take a look at Halide to HLS (https://github.com/jingpu/Halide-HLS), which is a similar system able to generate FPGA designs.  There's also Rigel (https://github.com/jameshegarty/rigel) which is a little lower level than Darkroom but was designed from scratch for generating FPGA designs.

@stevenbell Thank you!


Add dismiss swap to menu entry swapper

ignores Dunce & Genie since they reward xp
Co-Authored-By: Dava<EMAIL_ADDRESS>Added to the menu entry swapper. The Dismiss option is now the top option when the Dismiss option is checked (off by default). Applies to every random event other than the Dunce and Genie (xp giving events).

Thanks for committing this for me!

!!


Kafka message written to materialized view multiple times

I am testing the kafka engine. I have set up the following tables:
CREATE TABLE queue (
    eventtime DateTime,
    key String,
    value Float64
  ) ENGINE = Kafka SETTINGS kafka_broker_list = '<IP_ADDRESS>:9092',
                            kafka_topic_list = 'clickhouse',
                            kafka_group_name = 'clickhouse',
                            kafka_format = 'JSONEachRow',
                            kafka_num_consumers = 1;

CREATE TABLE data (
  eventtime DateTime,
  eventdate Date,
  key String,
  value Float64
)
ENGINE = MergeTree()
PARTITION BY toYYYYMM(eventdate)
ORDER BY (tag, eventtime)
SETTINGS index_granularity = 8192;

CREATE MATERIALIZED VIEW view TO data
    AS SELECT eventtime, toDate(eventtime) as eventdate, key, value
    FROM queue;

I then send a single message to the kafka topic:
{"eventtime": "2019-09-12 11:01:00", "key": "test", "value": 2.1}

I then end up with 139 rows in data, all repeats of the same message.
SELECT *
FROM data

┌───────────eventtime─┬──eventdate─┬─key──┬─value─┐
│ 2019-09-12 11:01:00 │ 2019-09-12 │ test │   2.1 │
│ 2019-09-12 11:01:00 │ 2019-09-12 │ test │   2.1 │
│ 2019-09-12 11:01:00 │ 2019-09-12 │ test │   2.1 │
│ 2019-09-12 11:01:00 │ 2019-09-12 │ test │   2.1 │
│ 2019-09-12 11:01:00 │ 2019-09-12 │ test │   2.1 │
│ 2019-09-12 11:01:00 │ 2019-09-12 │ test │   2.1 │
...

Logs:
2019.09.12 20:50:36.452721 [ 39 ] {} <Debug> StorageKafka (kafka_queue): Started streaming to 1 attached views
2019.09.12 20:50:36.453605 [ 39 ] {} <Trace> StorageKafka (kafka_queue): Subscribed to topics: clickhouse
2019.09.12 20:50:36.454090 [ 39 ] {} <Trace> StorageKafka (kafka_queue): Assigned to topics: clickhouse
2019.09.12 20:50:36.957547 [ 39 ] {} <Trace> StorageKafka (kafka_queue): Stalled
2019.09.12 20:50:36.960376 [ 39 ] {} <Trace> StorageKafka (kafka_queue): Committed offset 53 (topic: clickhouse, partition: 0)
2019.09.12 20:50:36.960580 [ 39 ] {} <Trace> StorageKafka (kafka_queue): Execution took 507 ms.
2019.09.12 20:50:37.461660 [ 23 ] {} <Debug> StorageKafka (kafka_queue): Started streaming to 1 attached views
2019.09.12 20:50:37.462734 [ 23 ] {} <Trace> StorageKafka (kafka_queue): Subscribed to topics: clickhouse
2019.09.12 20:50:37.463324 [ 23 ] {} <Trace> StorageKafka (kafka_queue): Assigned to topics: clickhouse
2019.09.12 20:50:37.965919 [ 23 ] {} <Trace> StorageKafka (kafka_queue): Stalled
2019.09.12 20:50:37.968827 [ 23 ] {} <Trace> StorageKafka (kafka_queue): Committed offset 53 (topic: clickhouse, partition: 0)
2019.09.12 20:50:37.969028 [ 23 ] {} <Trace> StorageKafka (kafka_queue): Execution took 507 ms.
2019.09.12 20:50:38.469332 [ 25 ] {} <Debug> StorageKafka (kafka_queue): Started streaming to 1 attached views
2019.09.12 20:50:38.469875 [ 25 ] {} <Trace> StorageKafka (kafka_queue): Subscribed to topics: clickhouse
2019.09.12 20:50:38.470066 [ 25 ] {} <Trace> StorageKafka (kafka_queue): Assigned to topics: clickhouse
2019.09.12 20:50:38.972128 [ 25 ] {} <Trace> StorageKafka (kafka_queue): Stalled
2019.09.12 20:50:38.975421 [ 25 ] {} <Trace> StorageKafka (kafka_queue): Committed offset 53 (topic: clickhouse, partition: 0)
2019.09.12 20:50:38.975639 [ 25 ] {} <Trace> StorageKafka (kafka_queue): Execution took 506 ms.
2019.09.12 20:50:39.477268 [ 24 ] {} <Debug> StorageKafka (kafka_queue): Started streaming to 1 attached views
2019.09.12 20:50:39.477824 [ 24 ] {} <Trace> StorageKafka (kafka_queue): Subscribed to topics: clickhouse
2019.09.12 20:50:39.478033 [ 24 ] {} <Trace> StorageKafka (kafka_queue): Assigned to topics: clickhouse
2019.09.12 20:50:39.981947 [ 24 ] {} <Trace> StorageKafka (kafka_queue): Stalled
2019.09.12 20:50:39.983935 [ 24 ] {} <Trace> StorageKafka (kafka_queue): Committed offset 53 (topic: clickhouse, partition: 0)
2019.09.12 20:50:39.984175 [ 24 ] {} <Trace> StorageKafka (kafka_queue): Execution took 506 ms.
2019.09.12 20:50:40.485669 [ 26 ] {} <Debug> StorageKafka (kafka_queue): Started streaming to 1 attached views
2019.09.12 20:50:40.486751 [ 26 ] {} <Trace> StorageKafka (kafka_queue): Subscribed to topics: clickhouse
2019.09.12 20:50:40.487242 [ 26 ] {} <Trace> StorageKafka (kafka_queue): Assigned to topics: clickhouse
2019.09.12 20:50:40.989686 [ 26 ] {} <Trace> StorageKafka (kafka_queue): Stalled
2019.09.12 20:50:40.992161 [ 26 ] {} <Trace> StorageKafka (kafka_queue): Committed offset 53 (topic: clickhouse, partition: 0)
2019.09.12 20:50:40.992368 [ 26 ] {} <Trace> StorageKafka (kafka_queue): Execution took 506 ms.

@den-crane That sounds like the same bug, but I am running ClickHouse server version 19.13.1 revision 54425.

I just did a docker pull and updated to ClickHouse server version 19.14.3 revision 54425. It seems to work as expected again on that version. Thanks.


moto/issues/2670 | Moved population of user attributes from accessToken to idToken

Also here, please run the command black --check moto/ tests/ locally to format your files and then the build will become green.

Coverage decreased (-0.2%) to 93.889% when pulling 13946100c07b02cf409ff3310991eb140d0ceeaf on franz-see:moto/issues/2670 into d596560971ae289f102b9aecb9939b3c49a56ac5 on spulec:master.

Thanks @gruebel . i've now pushed another commit fixing the formatting. This is now the output of black --check moto/ tests/
$ black --check moto tests
All done! ✨ 🍰 ✨
583 files would be left unchanged.


Design Get Involved Page (Mobile View/Responsive)

Figma (Mobile) In progress: https://www.figma.com/file/KbNIbEz1n8R5EVXEQLRSEX/Heat-Pump?node-id=2350-12722&t=hpFWWKUNjZ6qFpBE-0

Mobile/Responsive Frame (https://www.figma.com/file/KbNIbEz1n8R5EVXEQLRSEX/Heat-Pump?type=design&node-id=2350-12722&t=eioF6Tb1xmFF6MkG-0) Completed
@thiagobardini


Order type MarketOrder currently not supported by Catalyst

Dear Catalyst Maintainers,
I'm confused about how to use market orders. I can't figure out how order_target_percent wants me to input the style. If I use order_target_percent(context.market, 1, style=MarketOrder() I get Order type 'MarketOrder' currently not supported by Catalyst. Please use 'limit' or 'market' orders only. error. No variation seems to work. I tried to search the repo for examples but there doesn't seem to be anything.
Am I missing something here or doesn't Catalyst support market orders at all?

While I am unsure what is causing the issue with using style=MarketOrder() , I do not believe you need to do so.
I believe the default execution for order_target_percent is a market order, so using it without the a chosen execution style will execute a market order.