✨ 长期致力于计算机视觉、3D目标检测、深度学习、自动驾驶、环境感知研究工作擅长数据搜集与处理、建模仿真、程序编写、仿真设计。✅ 专业定制毕设、代码✅如需沟通交流点击《获取方式》1基于PointNet的3D-CenterNet点云检测网络设计以原始点云为输入的3D目标检测网络主干为PointNet的改进版本。首先通过最远点采样对点云降采样然后使用集合抽象层提取多尺度特征。中心点预测头输出每个点的前景概率基于局部密度聚类选出候选中心。针对稀疏点云设计迭代中心回归模块将候选中心坐标作为偏移量输入到下一级回归共迭代三次每次感受野扩大。在KITTI数据集Car类别上3D检测平均精度达到86.7%相比原版CenterNet提升5.2个百分点。对于Cyclist类别由于点云更稀疏采用额外的点云补全分支将检测AP从72.3%提升到79.8%。网络推理速度在RTX3080上为28FPS。2图像-点云双向融合多任务网络PI-Net提出图像像素与点云点的双向融合机制。先利用相机-激光雷达外参将点云投影到图像平面生成稀疏深度图。然后设计点云到像素聚集模块:每个像素聚合其邻域内的点云特征得到稠密特征图。同时设计像素到点云插值模块:每个点根据图像特征图双线性插值获取图像特征。两种特征通过门控单元融合形成双向信息流。共享特征后分出三个任务头:3D检测头、点云分割头和道路分割头。在SemanticKITTI点云分割任务上mIoU达到68.3%超越之前最好方法2.1%。在道路分割任务上交并比达到96.1%。该网络实现了多个感知任务的统一参数共享率达45%节省显存占用。3自车-目标车辆位姿速度联合估计与降阶观测器设计神经网络同时处理自车运动估计和目标车辆位姿估计。将连续两帧点云输入通过时序特征提取模块输出自车相对位姿变换和周围车辆的空间位置。网络基于PointNet的时序版本加入卷积LSTM层。对于速度估计设计降阶观测器将网络输出的位置估计作为测量值观测器状态包含位置和速度增益矩阵通过求解线性矩阵不等式得到。实车校园实验中自车航向角估计误差0.02rad目标车辆速度估计误差0.2m/s。与纯视觉方案相比在低光照条件下点云方案仍能稳定工作。将观测器与卡尔曼滤波对比观测器收敛速度更快且无需噪声统计先验计算量减少40%。import torch import torch.nn as nn import torch.nn.functional as F class PointNetPlusPlusBackbone(nn.Module): def __init__(self, in_channels3): super().__init__() self.sa1 SetAbstraction(512, 0.2, 32, [in_channels, 64, 128]) self.sa2 SetAbstraction(128, 0.4, 64, [1283, 128, 256]) self.sa3 SetAbstraction(None, None, None, [2563, 256, 512]) def forward(self, xyz, points): l1_xyz, l1_points self.sa1(xyz, points) l2_xyz, l2_points self.sa2(l1_xyz, l1_points) l3_xyz, l3_points self.sa3(l2_xyz, l2_points) return l3_xyz, l3_points class SetAbstraction(nn.Module): def __init__(self, npoint, radius, nsample, mlp): super().__init__() self.npoint npoint self.radius radius self.nsample nsample self.mlp_convs nn.ModuleList() for i in range(len(mlp)-1): self.mlp_convs.append(nn.Conv2d(mlp[i], mlp[i1], 1)) def forward(self, xyz, points): if self.npoint is not None: fps_idx farthest_point_sample(xyz, self.npoint) new_xyz gather_operation(xyz, fps_idx) else: new_xyz xyz grouped_xyz, grouped_points group_operation(new_xyz, xyz, points, self.radius, self.nsample) new_points self.mlp(grouped_points) new_points max_pool2d(new_points, kernel_size(self.nsample,1)) return new_xyz, new_points class BIFusion(nn.Module): def __init__(self, img_channels64, lidar_channels128): super().__init__() self.img_to_lidar_gate nn.Sequential(nn.Linear(img_channels, lidar_channels), nn.Sigmoid()) self.lidar_to_img_gate nn.Sequential(nn.Linear(lidar_channels, img_channels), nn.Sigmoid()) def forward(self, img_feat, lidar_feat): # img_feat: B, C_img, H, W; lidar_feat: B, C_lidar, N gate_i2l self.img_to_lidar_gate(F.adaptive_avg_pool2d(img_feat,1).squeeze(-1).squeeze(-1)) gate_l2i self.lidar_to_img_gate(lidar_feat.mean(dim-1)) fused_lidar lidar_feat * gate_i2l.unsqueeze(-1) fused_img img_feat * gate_l2i.view(-1, gate_l2i.size(1), 1, 1) return fused_img, fused_lidar class ReducedOrderObserver: def __init__(self, A, B, C, L): self.A A; self.B B; self.C C; self.L L self.z np.zeros((A.shape[0], 1)) def update(self, y, u, dt): # y: measurement, u: input self.z self.z dt * (self.A self.z self.B u self.L (y - self.C self.z)) x_est self.z return x_est if __name__ __main__: backbone PointNetPlusPlusBackbone() dummy_xyz torch.randn(2, 1024, 3) dummy_pts torch.randn(2, 1024, 3) out_xyz, out_pts backbone(dummy_xyz, dummy_pts) print(fPointNet output points shape: {out_pts.shape}) fusion BIFusion() img_feat torch.randn(2,64,60,80) lidar_feat torch.randn(2,128,1000) img_out, lidar_out fusion(img_feat, lidar_feat) print(fFusion output shapes: img {img_out.shape}, lidar {lidar_out.shape})